How about everyone just started following the CSV spec? https://tools.ietf.org/html/rfc4180 Doesn't allow for tab-delimited or any-character-delimited text and handles "Quotes, Commas, and Tab" characters in fields.
I love the way that, in the frickin' formal spec , the presence of a header row is ambiguous, so every tool that ever deals with CSV has to ask a human whether or not a header row is present. Great design decision, that.
ASCII Delimited Text – Not CSV or TAB delimited text
241–250 of 286 posts
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#242Why not use the non-printing char as the comma instead of the record separator. 1. Replace all the commas in the text with the unique non-printing char before converting to CSV. 2. Convert this char back to a comma when processing the CSV for output to be read by humans. Because commas in text are usually followed by a space, the CSV may still even be readable when using the non-printing char. I must admit I've never…
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#243Anyone that's ever had to parse arbitrary data knows of the approximately 14 jiggityzillion corner cases involved when sucking in or outputting CSV/TAB delimited formats. Yet much like virtual memory and virtual machines, we find that a solution has existed since the 60s. For those wondering about the history and use of all those strange characters in your ASCII table: http://www.lammertbies.nl/comm/info/ascii-charac…
Or just google for "ASCII table" and open the first hit: http://www.asciitable.com/ But your link does have more in-depth explanations (including historical info) for some of the control characters.
Just sayin'
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#244Re: ASCII Delimited Text – Not CSV or TAB delimited text
#245Earlier quoted context omitted.
I'm looking at a couple Macs sitting around the house and they all have a pretty obvious # right above the 3. Perhaps you mean some other key? (I do really miss the days when Mac keyboards had the weird symbol they use for "alt" in menus on the key.)
Which keyboard do you have? The standard aluminium full size one doesn't label it, at least in the uk
In the past, I've bought US Mac keyboards just for the #, or switched the keyboard layout in software to US.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#246Earlier quoted context omitted.
How would you tell…?
Ha. Only the Das Ultimate doesn't have printed keys. I got the Professional which has printed keys ( http://www.daskeyboard.com/model-s-professional/ ). I'm not that show-offy about touch-typing. ;)
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#247Earlier quoted context omitted.
Yes, but nowadays this is just another bootstrap problem: editors don't support them because no documents use them, no documents use them because editors don't support them, and users scream and bawl because their cheese has moved. Using an editor with some good support for control-character-separated data would be pleasant, more pleasant than the usual experience of fiddling with CSV in a text editor. That said, ano…
if the object is given a UUID, the recursive data structure can just point to the UUID when it's serialized to this? what's the problem?
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#248Earlier quoted context omitted.
you should never try to parse CSV yourself Why? Writing a correct parser is not significantly harder than figuring out how to interface to an existing parser library, and allows cool things like heuristic parsing of malformed files. OTOH it's shocking how many people can't write a correct CSV generator , even after being explicitly told what they're doing wrong (which is always either "you need to put quotes around t…
If you knew enough about CSV to be able to write a correct parser, then you'd know enough not to write one lightly. Here are some surprising valid CSV files: https://forge.ocamlcore.org/plugins/scmgit/cgi-bin/gitweb.cg... The test program in the same directory shows the semantic content of each.
The RFC declares itself "informational" and says things like there is no formal specification in existence, which allows for a wide variety of interpretations of CSV files. This section documents the format that seems to be followed by most implementations:
There are certainly reasonable arguments about the useful subset of rules.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#249Earlier quoted context omitted.
Which keyboard do you have? The standard aluminium full size one doesn't label it, at least in the uk
On US Mac keyboards, # is Shift-3. On UK Mac keyboards Shift-3 is the Pound (Sterling) sign. # becomes Alt-3 and is unlabelled on the keyboard. In the past, I've bought US Mac keyboards just for the #, or switched the keyboard layout in software to US.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#250It is now 2014. The world doesn't use ASCII, you will still need escaping for binary or misformatted data, and overall the idea of mapping control characters and text into one space is dead and dusted . Don't do it, don't let other people do it, use a reasonable library that handles the bazillion edge cases safely if you need to parse or write CSV and its ilk.
Due to the volume and novelty of data that we work with, we are often pushed into a corner between human time and machine time. Each data set comprises a new set of concepts, and each is huge. In this corner, sometimes a character-delimited file is the best solution. There is not time to carefully craft a binary format and then document it so it will not be forgotten later, nor is there time to wait for a general-purpose format parser to operate on tens of billions of records. We need a solution that can be designed in 1 minute and be legible by all of our tools without modification.
Typically, I have used tabs in the place of the ASCII separators. This ensures readability without any kind of parsing. Also, this lets me use the default behaviors of well-worn, bug-free tools in the core of the Unix toolchain for basic data processing tasks. Frankly, this is not a bad compromise.
If you are passing messages around a web stack, JSON, XML, and friends are ideal solutions. If you have to occasionally deal with CSV, use a parser. I just want to note that for many tasks in data analysis, it's OK to simply use the dead and dusted convention of mixed delimiters and data.
As these things develop, I will be trying to investigate how to use more modern formats such as binary JSON representations in my work, and I'd be curious what solutions people here suggest for working with very large data (e.g. many trillions of observations).