Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ASCII and Unicode quotation marks

cl.cam.ac.uk

Post reply on HN
Print view View original

81–90 of 195 posts

Re: ASCII and Unicode quotation marks

#81

Post by moron4hire »

kps wrote: post #33
cesarb wrote: post #29

Earlier quoted context omitted.

From what I recall from my childhood, physical typewriters worked slightly differently: the accent keys were non-advancing ("dead") keys. You pressed the "acute" key followed by the "e" key for an é, for instance. If you wanted a bare accent, you pressed the accent key followed by the space bar. (The typewriters I recall also didn't have a 0 or 1 key, you used uppercase O or I for these numbers.)

Yes. I consider it a mistake of Unicode that combining characters follow rather than precede the base character. If they preceded, most dead keys could simply generate the appropriate combining character, rather than requiring complicated input method support. (And finding the end of a sequence of multiple combining character wouldn't require lookahead.)

The Unicode way makes sorting easier. Your way would require special knowledge about the characters to know that ä should sort directly after a, rather than directly before ë.

Re: ASCII and Unicode quotation marks

#82

Post by spc476 »

emmelaich wrote: post #75
diamondo25 wrote: post #71

Earlier quoted context omitted.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

If it's for my own programs, I use pipe (|) separated values. They're visually appropriate and even less likely than tabs.

ASCII does contain control characters set aside for record and unit separators (codes 30 and 31 respectively). Sometimes I wish they got more use than they do.

Re: ASCII and Unicode quotation marks

#83

Post by panic »

PickNChoose wrote: post #67
lisper wrote: post #53

Earlier quoted context omitted.

> You would still need an escape mechanism for something like... Yes, but that's a pretty rare case, much more so than embedded strings. Even that case could be solved by having two different quotes, like Python which allows both 'string' and "string". So you could do: «This is a string that mentions the ” character without escaping it» “This is a string that mentions the « character without escaping it” Yes, there a…

You don't want any 'rare' cases at all. That's the point. Stop using "punctuation" when you are attempting to "delimit" text. Use a character that is not punctuation, specifically designed for "field delimiter" purposes. Trying to do two things at once is ridiculous.

If your text is always valid UTF-8, there are various illegal UTF-8 octets available for this purpose: 0xff, 0xfe, and so on. Unlike null terminators or record separator characters, these characters are guaranteed not to exist in your string by the UTF-8 validation code you're already running.

Re: ASCII and Unicode quotation marks

#84

Post by psadauskas »

treve wrote: post #10

It just occurred to me how much easier certain text-operations (like syntax highlighting, regular expressions and other parsers) if we consistently used the right unicode symbols for quotes and apostrophes

And every time I get in an argument with a poorly-escaped CSV file, I wish we had just used ASCII 28-31 as delimiters. (File, Group, Record and Unit Separator)

Re: ASCII and Unicode quotation marks

#85

Post by Someone »

hk__2 wrote: post #58
tome wrote: post #50

Earlier quoted context omitted.

If ASCII had balanced quotes then they would be used by programming languages to delimit strings and we would be back to square one with regards to escaping them!

You don’t need escaping in «This is a string containing an «embedded» quoted string».

So, how would you encode this in a string:

  To end a string, use the » character.

?

Re: ASCII and Unicode quotation marks

#86

Post by jl6 »

diamondo25 wrote: post #71
PickNChoose wrote: post #67

Earlier quoted context omitted.

You don't want any 'rare' cases at all. That's the point. Stop using "punctuation" when you are attempting to "delimit" text. Use a character that is not punctuation, specifically designed for "field delimiter" purposes. Trying to do two things at once is ridiculous.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

What's the problem you're having with Excel reading TSV? Works fine here.

Re: ASCII and Unicode quotation marks

#87

Post by Avshalom »

jl6 wrote: post #86
diamondo25 wrote: post #71

Earlier quoted context omitted.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

What's the problem you're having with Excel reading TSV? Works fine here.

Most *SV importers will actually accept any character as the delimiter, it's just people insist on believing CSV is utterly trivial and thus not worth using a real library for it.

Re: ASCII and Unicode quotation marks

#88

Post by contingencies »

ansgri wrote: post #49
dmitriid wrote: post #36

It's worse for other languages. Russian quotation marks are « and ». Thanks to early computers being predominantly from/designed in the US, they are now highjacked by American quotes. Same probably goes for French and other languages with their own sets of quotation marks.

«Russian» quotation marks are actually the « French » ones with different spacing. There's another, less used set of quotes in Russian, so called „German“ ones (used as inner quotes and in handwriting). English quotes are widely accepted though.

Modern Chinese usage includes all of 《》〈〉「」『』【】“” and probably others, roughly in that order. Modern typographic convention is perhaps 《title》「quote」 but that's surely opine and debatable. Hong Kong and Taiwan have their own typesetting conventions, distinct from mainland China, and in the latter case no doubt influenced by Japanese occupation and cultural inflow (manga, etc.). Historically for most of Chinese history written language had no punctuation, and sentence endings were merely inferred from context, which was historically clearer 也. See https://en.wikipedia.org/wiki/Chinese_punctuation and https://en.wiktionary.org/wiki/%E4%B9%9F#Definitions (definition #4)

Re: ASCII and Unicode quotation marks

#89

Post by gumby »

YSFEJ4SWJUVU6 wrote: post #61
gumby wrote: post #38

I find it interesting that the article includes a German keyboard that doesn't include the proper ,,'' (or ,') quotation glyphs. However it does include grave and acute accents as well as French primary quotations ( >) though not the secondary guillemots (quotation characters ) none of which are used in German text. And of course I used ascii analogues to type these into HN :-(

>And of course I used ascii analogues to type these into HN :-( But why, though? To the best of my knowledge, HN supports unicode quite well, including the following quotes: »«›‹„“‚‘ (available with the help of AltGr and sometimes shift from keys y, x, v, b when selecting the German keyboard layout on my computer).

I'm using a travel laptop on a plane and it came with a US keyboard

Re: ASCII and Unicode quotation marks

#90

Post by ttepasse »

The usage of of an accent as syntax in markup and programming languages annoys me to no end. And it will still be used, to this day, the latest example are template string in Javascript.

• It is semantically idiotic because it's an accent, not a character.

• It is visually annoying because you almost can't see the thing.

• It is bad for usability, because on non-US keyboards the accents are implemented as dead keys. Yes, accent + space gives you the character but that's really unintuitive for people who grew up expecting accents only over letters.

Post reply on HN