Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ASCII and Unicode quotation marks

cl.cam.ac.uk

Post reply on HN
Print view View original

101–110 of 195 posts

Re: ASCII and Unicode quotation marks

#101

Post by alanh »

ComputerGuru wrote: post #92
diamondo25 wrote: post #71

Earlier quoted context omitted.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

> but text can contain commas Erm.. text can contain tabs, too. This problem was solved so, so long ago when all the various ANSI/ASCII/whatever encodings were compiled by specifically reserving not one but two characters precisely to serve as field and record delimiters. 0x30 and 0x31 solved not only the problem of having commas or tabs in your text preventing you from treating them as field delimiters, but also all…

I have to suspect the fatal flaw is that these code points don’t look like anything and can’t be found on the keyboard.

Granted, that’s the whole point, but it also makes authoring and instruction harder. (And we all know how many programmers are really just competent copy-pasters.)

Re: ASCII and Unicode quotation marks

#102

Post by Pxtl »

ComputerGuru wrote: post #92
diamondo25 wrote: post #71

Earlier quoted context omitted.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

> but text can contain commas Erm.. text can contain tabs, too. This problem was solved so, so long ago when all the various ANSI/ASCII/whatever encodings were compiled by specifically reserving not one but two characters precisely to serve as field and record delimiters. 0x30 and 0x31 solved not only the problem of having commas or tabs in your text preventing you from treating them as field delimiters, but also all…

Yes, but at that point the text file is basically binary - it contains exotic characters that confuse most text editors and can't be typed.

I know XML et al are frustrating, but I'd rather see them than a "creative" solution. It seems like 60% of the reason we still have to deal with archaic flat formats is support for Excel.

Re: ASCII and Unicode quotation marks

#103

Post by alanh »

stephengillie wrote: post #70
dragonwriter wrote: post #62

Earlier quoted context omitted.

Also Ruby: %q{This is a string with an %q{embedded quote}.}

Powershell: "This is a string with an 'embedded quote'." It's helpful to remember that quotes will interpret the variables inside, while apostrophes will not. Very useful for scripting the creation of scripts. Example: "It is $time" > It is 15:22 'It is $time' > It is $time "'$time' is $time" > '$time' is 15:22

not sure why you were downvoted. Your comment is relevant and the convention can be useful. (PHP worked exactly the same way as your first two examples, although the third would have produced .)

Re: ASCII and Unicode quotation marks

#104

Post by Pxtl »

emmelaich wrote: post #75
diamondo25 wrote: post #71

Earlier quoted context omitted.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

If it's for my own programs, I use pipe (|) separated values. They're visually appropriate and even less likely than tabs.

Except pipe is really easy to get as a typo. It's right next to the enter key. And then you're dealing with escaping characters and before you know if you've rolled your own file format.

Been there. Use a lib that implements a documented standard, even a bad one. Only problem is Excel, which basically standardizes on CSV and occasionally mangles your data into malformed dates because reasons anyways.

Re: ASCII and Unicode quotation marks

#105

Post by alanh »

ComputerGuru wrote: post #96
leephillips wrote: post #37

Earlier quoted context omitted.

Nobody's keyboard had an em-dash key (well, maybe some compositor keyboards...). It's well worth the time to figure out how to enter at least the most useful Unicode characters. See, here's an em-dash: —.

I use the code behind this web app: http://latex2unicode.herokuapp.com/ It converts latex to unicode where possible. It's pretty impressive how much of Latex can be replaced with Unicode today. This program translates LaTeX markup to human-readable Unicode when possible. Here's the default text from that webapp: Basic math notations: ∵ A͡B + B͡C ≠ A͡C ∴ ∬∜x̅ ξᶿ⁺¹ - ⅜ ≤ Σ ζᵢ ∴ ∃x∀y x ∈ Â Easily type in hundreds of ot…

Careful — some of the Unicode pseudoalphabets won't render on mobile.

Re: ASCII and Unicode quotation marks

#106

Post by Pxtl »

drdeca wrote: post #59
hk__2 wrote: post #58

Earlier quoted context omitted.

You don’t need escaping in «This is a string containing an «embedded» quoted string».

I think it would make the parsing a tiny bit more inconvenient though

Yup, suddenly your parser needs to keep a count of how many nested strings deep it is.

Re: ASCII and Unicode quotation marks

#107

Post by alanh »

gbacon wrote: post #26
gbacon wrote: post #24

Earlier quoted context omitted.

Another giveaway of a TeX-savvy writer out of water is when you see --- for em-dash, i.e. , ‘—’.

Does HN markdown understand — or ‘? EDIT: Nope.

not that HN needs more pedantry, but HN’s lightweight markup format is not in any sense a Markdown. I believe that literally the only thing they have in common is that a single set of asterisks yields italics. Bolding, headings, code, lists, quotes, links, etc. don’t transfer from one to the other.

Re: ASCII and Unicode quotation marks

#108

Post by Pxtl »

Symbiote wrote: post #51
agumonkey wrote: post #47

Earlier quoted context omitted.

let's rewrite social idioms to use as quotes.

«These characters» are the usual way of quoting in several languages. See https://en.wikipedia.org/wiki/Guillemet

Well then it's a good thing c used its ASCII equivalent that's accessible on anglophone keyboards for bit shifting and so if any programming language tried to use > you'd have c grognards screaming about lshift and rshift.

Re: ASCII and Unicode quotation marks

#110

Post by stormbrew »

eesmith wrote: post #48
lisper wrote: post #41

The fact that ASCII does not have balanced quotes is one of the great catastrophes of computing. It makes everything more complicated than it needs to be, from embedding code in strings to parsing CSV files, to regexps. For example, if I want to embed a quoted string in another quoted string, I have to escape the inner quotes like so: "This is string containing an embedded \"quoted\" string" Then I have to think abou…

The complexity might be minimized, but not avoided. You would still need an escape mechanism for something like «She said «The \» key on the server doesn't work.»» ASCII did add , [], and {}, any of which could have been used for quoted strings, had the programming language designers chosen that option. https://en.wikipedia.org/wiki/String_literal#Paired_delimite... points out that PostScript and Tcl have a string li…

Ruby lets you use arbitrary tokens for string literals with %s{} (where the braces can be a bunch of things). I wish more languages would adopt this tbh.
Post reply on HN