ASCII Delimited Text – Not CSV or TAB delimited text

Post by jimmaswell » Wed, Mar 26, 2014, 8:21 PM UTC

leoc wrote: post #36

Pxtl wrote: post #10

I've done this. Everybody hated it. Most text editors don't display anything useful with these characters (either hiding them altogether or showing a useless "uknown" placeholder), and spreadhseet tools don't support the record separator (although they all let you provide a custom entry separator so the "unit" separator can work). Besides the obvious problem that there's no easy way to type the darned things when som…

Yes, but nowadays this is just another bootstrap problem: editors don't support them because no documents use them, no documents use them because editors don't support them, and users scream and bawl because their cheese has moved. Using an editor with some good support for control-character-separated data would be pleasant, more pleasant than the usual experience of fiddling with CSV in a text editor. That said, ano…

Post by wtallis » Wed, Mar 26, 2014, 8:36 PM UTC

derefr wrote: post #94

Someone wrote: post #84

Earlier quoted context omitted.

Even ignoring additional time needed to send those extra characters, it also was a lot faster than those control-H's, and (I guess) caused way less wear on your printer.

That presumes you're going like A^HAB^HB; you could just emit foo, and then strlen(foo) ^Hs.

Post by groovy2shoes » Wed, Mar 26, 2014, 8:46 PM UTC

npongratz wrote: post #71

scelerat wrote: post #44

Earlier quoted context omitted.

ASCII 0-31 are called "control" characters so it should come as no surprise that you could type them using the Control key. Unit Separator is Control-_ (underscore) and Record Separator is Control-^ (caret), for instance. Most modern text editors won't pass through every control character. Vim lets me type the unit separator, but not the record separator, for instance. Control-C and Control-D, End of Text and End of…

Vim will let you type the record separator, but you may have to precede it with Ctrl-V. In other words, type the Ctrl-V Ctrl-^ key sequence. At least, that worked for me. Interesting that the unit separator does not have the same requirement to precede it with Ctrl-V. More info in the vim docs: http://vimdoc.sourceforge.net/htmldoc/insert.html#ins-specia...

Post by radicaledward » Wed, Mar 26, 2014, 8:47 PM UTC

dxbydt wrote: post #57

Don't do this. Tsv has won this race, closely followed by Csv. Anything else will cause untold grief for you and fellow data scientists and programmers. I say this as someone who routinely parses 20gb text files, mostly Tsv's and occasionally Csv's for a living. The solution you are proposing is definitely superior but isn't going to get adopted soon.

Post by danielweber » Wed, Mar 26, 2014, 8:52 PM UTC

sopooneo wrote: post #95

raldi wrote: post #79

Earlier quoted context omitted.

...and that's why, when running a command that's reading what you're typing to stdin, you press Ctrl-D to tell it that you're done. Because, as your "man ascii" trick shows, this generates the "end of transmission" character. Similarly, Ctrl-L clears the screen in most Unix apps because that generates the "form feed" character, and on a line printer or paper-based teletype terminal, "form feed" means to advance to th…

what about Ctrl-C to break out of stuff? Does that correspond to an ascii control character?