21–29 of 29 posts
I think we should name the files ".cso" CSO is a stormwater industry term for "Combined Sewer Overflow." They happen in older cities where storm runoff and raw sewage (poop) go into the same sewer system. When there is a lot of rain, the wastewater treatment plants overflow, and then raw sewage runs into waterways. https://en.wikipedia.org/wiki/Combined_sewer#Combined_sewer_...
I think we should name the files ".cso" CSO is a stormwater industry term for "Combined Sewer Overflow." They happen in older cities where storm runoff and raw sewage (poop) go into the same sewer system. When there is a lot of rain, the wastewater treatment plants overflow, and then raw sewage runs into waterways. https://en.wikipedia.org/wiki/Combined_sewer#Combined_sewer_...
Earlier quoted context omitted.
Because those of us that have to read your data would highly prefer you just emit standard¹ CSV, and not invent "CSV+my oddball customizations". If you're going to muck about outside the standard format, then you might as well just use DSV from the OP. Most good implementations are flexible enough that they might be configurable to your proposed pseudo CSV. (Or even DSV. Or USV. Etc.) But I'd rather just not need to,…
"Oddball customisation" is a bit rich, no? A backslash is the way things are escaped in most places. Why re-invent the wheel? And why would you "highly prefer you just emit standard CSV"? What is the benefit to insisting adherence to the original standard, especially if the modification fixes something that is broken?
Sure … but not by CSV. Backslash is hardly the only way, nor is the doubled-quote escape mechanism particularly obscure, given its presence in popular formats like CSV, SQL, or YAML.
> why would you "highly prefer you just emit standard CSV"?
Why would you prefer a baroque format?
So that it goes through a standard parser without needing extra, additional configuration. It's just CSV, and moreso, it's CSV without hoops or surprises that need to be transmitted out of band somehow.
> What is the benefit to insisting adherence to the original standard, especially if the modification fixes something that is broken?
That's not the case here, though: using backslash doesn't fix something that's broken.
It's a problem solved decades ago with solutions we've failed to adopt. Weird, buggy, poorly parsable CSV is still somehow the norm.
Not saying you should, but if you want to change, the answer is already there. Change has to start somewhere...
0x1d and 0x1e in the ascii standard exist for exactly this reason and don’t need more than one byte unlike this goofy thing.
0x1d and 0x1e in the ascii standard exist for exactly this reason and don’t need more than one byte unlike this goofy thing.
Why do the RS and GS characters get ignored so much? I have only seen them used in symbols on parts labels of one of the big 3 automakers.
No! The poop symbol is used in data, and thus is a terrible separator. If you have to quote it anyway, use commas, as that is already in use. Or use "Unicode" separated values.
You could just apply the standard CSV escaping mechanism, so a single poop in a cell is represented in the file by 4 poop emoji. (Two to quote, one to escape, one as a literal. See RFC 4180 if you're still confused on "why 4?")
What if my data contains a new line? People focus on the comma then forget the newline is just as significant. That still needs to be escaped and we're right back where we started. Meanwhile, RFC4180 takes less time to read than this entire article.
CSV is a simple storage format for data. Its simplicity, readability and portability makes it popular. I think that any attempt to improve it will be a failure.
I must say that CSV generally suffices for table data. The only annoyance is that internationally, there are differences in the use of the column separator, as the comma is often used as decimal separator. I think CSV should always be implemented with a comma as column separator and a dot as decimal separator, regardless of the country. But applications such as Excel do not accept this format internationally.