Live data from Hacker News

ASCII and Unicode quotation marks

cl.cam.ac.uk

131–140 of 195 posts

Re: ASCII and Unicode quotation marks

#131
post #30

to add to the confusion: ' PRIME (U+2032) " DOUBLE PRIME aka inch mark (U+2033) have their own codepoints http://practicaltypography.com/foot-and-inch-marks.html which describes implications for typesetting coordinates and other things: 118° 19′ 43.5″ 118° 19’ 43.5” wrong (curly quotes, although it renders identical in some fonts) 118° 19' 43.5" right

And further confusion:

An ʻokina (U+02BB, as found in "Hawaiʻi") is neither an apostrophe nor a left quotation mark.

https://en.wikipedia.org/wiki/%CA%BBOkina

Re: ASCII and Unicode quotation marks

#132
I hate the "" -> “” thing with a passion. I don't know how much productivity that the world has lost with that “” shit.

It doesn't look that much better, and it always fucks with me at random times. That shit is on the list of annoying problems that shouldn't exist in the first place, along with the \nl\cr thing, and the txt saved as rtf thing, and the UTF-8 encoding-character-at-the-beginning-of-the-file or whatever it is called.

Someone complains your program gives them an error when they open a csv file you sent them. You tested your program, it works. You go on the phone with them for 30 minutes, try to figure out what the fuck was going on. There it is, it was opened in a program that meddles with that "" and replaces it with the “” shit.

Also, there has to be at least one time you're fucked by the "" -> “” snobbiness when you go to a random Wordpress site and paste the command they tell you to do to the command line and realize it doesn't work. You pull your hair for a couple of minutes, and there is that sneaky ” thing. Wordpress does that for anything it doesn't think as code (inb4 ”good programmers don't paste commands from wordpress to GNU+bash“).

One of the first things that I do when I set up a new Mac computer is to turn that damn "" -> “” ““““feature”””” off.

Re: ASCII and Unicode quotation marks

#133

I hate the "" -> “” thing with a passion. I don't know how much productivity that the world has lost with that “” shit. It doesn't look that much better, and it always fucks with me at random times. That shit is on the list of annoying problems that shouldn't exist in the first place, along with the \nl\cr thing, and the txt saved as rtf thing, and the UTF-8 encoding-character-at-the-beginning-of-the-file or whatever…

> at-the-beginning-of-the-file

That thing's the BOM.

Re: ASCII and Unicode quotation marks

#134
post #4

I would fain use the curly quotes if only Darwin's groff(1) wouldn't barf on them. For the time being, man pages for one still need to quote like ``this''.

In troff you can escape “ ” ‘ ’ as \(lq, \(rq, \(oq, \(cq respectively.

If you’re writing manpages, though, you should be using the -mdoc macros (https://manpages.bsd.lv/mdoc.html), which have “Dq” and “Sq” macros that wrap the arguments in double and single quotes respectively.

Re: ASCII and Unicode quotation marks

#135

I hate the "" -> “” thing with a passion. I don't know how much productivity that the world has lost with that “” shit. It doesn't look that much better, and it always fucks with me at random times. That shit is on the list of annoying problems that shouldn't exist in the first place, along with the \nl\cr thing, and the txt saved as rtf thing, and the UTF-8 encoding-character-at-the-beginning-of-the-file or whatever…

You forgot to mention leading apostrophes getting autocorrected to left_single_quote instead of right_single_quote, as in

John Doe ‘42

An abomination ‘cause it's a damn apostrophe, not opening a quotation.

Re: ASCII and Unicode quotation marks

#136
post #97

CSB: Years ago I was working on a team that developed a scripting language and we had this recurring problem where someone would write up a code sample in a Word document and it would break if you cut and pasted it because all of the single and double quotes would be Unicode. My boss was this tough guy who tried to snap the whole team to a standard of strictly disabling that behavior in all of our Office applications…

[deleted]

Re: ASCII and Unicode quotation marks

#137

Earlier quoted context omitted.

You don't want any 'rare' cases at all. That's the point. Stop using "punctuation" when you are attempting to "delimit" text. Use a character that is not punctuation, specifically designed for "field delimiter" purposes. Trying to do two things at once is ridiculous.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

Excel will convert any tabulated text file into a spreadsheet regardless of the delimiters, or even lack of, as you can set which character(s) to delimit by or even just go by column numbers for tables of fixed widths. This is actually one of the few things Excel gets right with regards to CSV files as I've found it a horrid tool if you need to save any changes and preserve the original formatting of the CSV file (even the data itself gets altered!!)

Also most CSV parsers support quotation marks and escaping to get around the comma and new line et al problems. eg:

    "full name", "address"
    "Homer Simpson", "742 Evergreen Terrace,\nSpringfield"
    "Bart \"El Barto\" Simpson", "742 Evergreen Terrace,\nSpringfield"
Granted it's not the prettiest and some spreadsheets really love to break the formatting upon save (cough Microsoft Excel cough) but it does work.

As a side note, the best spreadsheet I've found for manipulating CSV data without breaking the formatting upon saving was OpenOffice Calc. This was a few years back before the LibreOffice fork was created as I've thankfully not needed to deal with CSV files large enough to warrant a full blown spreadsheet editor, but I would assume LibreOffice Calc would behave the same.

Re: ASCII and Unicode quotation marks

#138
What bothers me about Unicode isn't that apostrophe (U+0027) is overloaded by having two semantic meanings ("apostrophe" or "single straight quote"), but that they exacerbate the confusion by recommending to overload "right single quote" (U+2019) to also mean apostrophe.

We now have two characters for apostrophe and extra ambiguity for processing correct right single quotes. Great job not breaking historical documents Unicode.

Re: ASCII and Unicode quotation marks

#139

I hate the "" -> “” thing with a passion. I don't know how much productivity that the world has lost with that “” shit. It doesn't look that much better, and it always fucks with me at random times. That shit is on the list of annoying problems that shouldn't exist in the first place, along with the \nl\cr thing, and the txt saved as rtf thing, and the UTF-8 encoding-character-at-the-beginning-of-the-file or whatever…

For anyone wondering, this is how to turn that feature off.

System Preferences -> Keyboard -> Text -> Use smart quotes and dashes (uncheck).

Re: ASCII and Unicode quotation marks

#140
post #41

The fact that ASCII does not have balanced quotes is one of the great catastrophes of computing. It makes everything more complicated than it needs to be, from embedding code in strings to parsing CSV files, to regexps. For example, if I want to embed a quoted string in another quoted string, I have to escape the inner quotes like so: "This is string containing an embedded \"quoted\" string" Then I have to think abou…

I disagree. Sure it might regex better, but my typing speed and typo rate would be much worse if I had to type separate open and close quotes for all my strings.
Post reply on HN