How do you enter them ? in console, in editor?
Since they are invisible, how do you find if you have entered a wrong character?
ASCII Delimited Text – Not CSV or TAB delimited text
11–20 of 286 posts
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#12Alas, I don't think this works with the standard Unix tools, which is the main way I process tab-delimited text. Changing the field delimiter to whatever you want is fine, since nearly everything takes that as a parameter. But newline as record separator is assumed by nearly everything (both in the standard set of tools, and in the very useful Google additions found in http://code.google.com/p/crush-tools/ ). Google'…
Awk lets you set both:
$ echo -n "1,2,3|4,5|6|7,8,9,0" | awk 'BEGIN{FS=","; RS="|"} {print NF, $0}'
3 1,2,3
2 4,5
1 6
4 7,8,9,0
$ echo -n "1,2,3|4,5|6|7,8,9,0" | awk 'BEGIN{FS=","; RS="|";OFS="foo";ORS="bar"} {print NF, $0}'
3foo1,2,3bar2foo4,5bar1foo6bar4foo7,8,9,0barRe: ASCII Delimited Text – Not CSV or TAB delimited text
#13It doesn't solve the problem, although it does make it far less likely to run into it. For a trivial example, try building an ASCII table using this format, with columns for numeric code, description, and actual character. You'll once again run into the whole escaping problem when you try to write out the row for character 31.
For CSV forbidding commas in data is not practical.
For ASCII delimiters, forbidding ASCII delimiters in data is practical.
Sure - you can't, say, nest ASCII tables into one another due to this limitation.
But for simple structure, it doesn't hurt to have ASCII separators in the toolbox.
The only big problem I see is that they're rendered as invisible characters, which will make debugging harder. If we wouldn't have abandoned and forgotten the special ASCII chars, this wouldn't be the case.
If your dev tools show special chars (like mine do), then it's perfectly fine to use them.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#14It doesn't solve the problem, although it does make it far less likely to run into it. For a trivial example, try building an ASCII table using this format, with columns for numeric code, description, and actual character. You'll once again run into the whole escaping problem when you try to write out the row for character 31.
Sure, but this is a very special application. whitespace and '"' are much more common in normal text, so using dedicated characters for telling entries apart should be superior in almost all cases. (The problem reminds me of what it's like using "/" as a delimiter when using `sed` to edit file paths.)
Right, but the "almost all" gives me pause.
I'm wary of anything that solves a problem partially while still remaining vulnerable in the end, because it can discourage properly solving the problem. Rather than play musical chairs with the separator character to try to minimize the chance of a conflict, I'd rather see a sensible encoding/escaping scheme used to eliminate that chance entirely.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#15This reminds me of a depressing bug I run into frequently. I do a bit of work integrating with an inventory management program. Their main method of importing/exporting information is via CSV. The API also imports and exports via CSV, except whoever wrote the code that handle the imports decided not to use any sort of sensible library. Instead they use a built-in function that splits the string based on commas with absolutely no way of escaping, so that there is no way to include a comma in a field.
It's led to many a headache.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#16"Alright let me get you some quick test data. Just need to find the 0x29 key on my keyboard... or 0x30? Wait is this a new row or a new column? What was the vim plugin for this?"
And then someone wrote an open source CSV parsing library that handles edge cases well and everyone forgot these characters existed.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#17My guess is that TSV/CSV won out simply because anyone can easily type those characters from any standard keyboard on any platform.
[deleted]
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#18This reminds me of a depressing bug I run into frequently. I do a bit of work integrating with an inventory management program. Their main method of importing/exporting information is via CSV. The API also imports and exports via CSV, except whoever wrote the code that handle the imports decided not to use any sort of sensible library. Instead they use a built-in function that splits the string based on commas with a…
I deal with a vendor who occasionally sends us files without the double-quote character escaped. I feel your pain.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#19The big problem with the two markers mentioned in the post is they are not part of the visible character set. Using a comma delimiter is good as it is visible, you can just use a basic text view to see it.
A tab delimiter is not preferable as it is not visible, and can be problematic to parse via command line tools (ie what do I set as the delimiter character?).
I think that is the whole point of having ASCII delimited text files is to have human readable data in it.
Re: ASCII Delimited Text – Not CSV or TAB delimited text
#20How do you enter them ? in console, in editor? Since they are invisible, how do you find if you have entered a wrong character?
All valid questions, all valid reasons they never caught on.