251–260 of 355 posts
Earlier quoted context omitted.
Most csv utilities support an alternative delimiter. If I need to edit a file by hand, I'll typically pick an uncommon character for the delimiter (pipe "|" works well since it's uncommon). For me, that pretty much entirely eliminates any of the pain with CSV.
tab-separated-value has never betrayed me! I think it's the default postgresql file export too.
Earlier quoted context omitted.
parquet is great but it's not particularly easy to read or write. the libraries that do exist to work with it are few and far between, and those that do either have a hundred dependencies or depend on native code (e.g. libarrow). certainly an important dimension in an ideal file format should be the ease of parsing/writing it, and parquet gets an extremely low score on that front imo
Parquet is also column-major which is great for many use cases, but bad for others, where row-major is more useful. For example, if you want to get just the first x rows.
The proposal is TSV but using ^_ instead of ^I (tab) to separate columns and ^^ instead of ^M (CR) or ^J (LF) to separate rows. So instead of being unable to put ^I and ^J in your table cells you are now unable to put ^_ and ^^ in your table cells. That sounds exactly as good, or bad, as TSV. So, okay? Sure, do that if you like, that sounds fine. Emacs TAGS files, Info files, and BABYL mailboxes https://quimby.gnus.o…
Also I believe this was the original purpose for US and RS in ASCII. So I'm not sure it is 'weird' to use them for that (although it does seem to be uncommon).
Earlier quoted context omitted.
I've seen ubiquitous use of tab-separated value files instead of csv, as as simpler format without quoting support and a restriction that your data fields can't contain tabs or newlines, which (unlike commas) is okay for many scenarios.
But that’s sort of the problem with csv. You never really know which rules your csv files has. Many .csv files are indeed tab separated.
Earlier quoted context omitted.
But that’s sort of the problem with csv. You never really know which rules your csv files has. Many .csv files are indeed tab separated.
Gotta wonder why the format isn't just a column separator char, a row separator char, and then all the data guaranteed not to have those two chars. Then you could save the thing by finding any two chars that aren't used in the data. I guess this is why we have a zillion formats.
ASCII Code 30: Record Separator
ASCII Code 31: Unit Separator
https://theasciicode.com.ar/ascii-control-characters/record-...
Earlier quoted context omitted.
Gotta wonder why the format isn't just a column separator char, a row separator char, and then all the data guaranteed not to have those two chars. Then you could save the thing by finding any two chars that aren't used in the data. I guess this is why we have a zillion formats.
I guess if you use an strange character nobody has on their keyboard, you could also just make it a binary format for efficiency
> Why can’t we have a format where Does Excel support it? No? Then that's the end of that. Excel is tabular data to all non developers. The formats supported by Excel are the whole thing. And if we're inventing a CSV-like format that uses a more convenient character than quotes and commas, maybe jumping to a non-displayable non-typeable character isn't the best? Honestly, if I were inventing a table format, I'd use a…
Here's the GitHub repo for what I like to call HSV5: https://github.com/elcritch/hsv5/blob/main/README.md ;)
And an example of the format, pretty similar to yours:
sensor data @ 2020-05-26T00:00:00.000Z
Timestamp
Temperature
Humidity
Pressure
2020-05-26T00:00:00.000Z8.3275.5102073
2020-05-26T00:01:00.000Z8.31NaN102074
2020-05-26T00:02:00.000Z8.3175.4102074
2020-05-26T00:03:00.000Z8.3075.4102074
Seems like the problem here is there is several high quality and well-developed formats, but the author and the commenters here dismiss them because of the different trade-offs they make. csv -- Simple for simple use cases, text-based, however many edge cases, feature lacking etc xlsx -- Works in excel, ubiquitous format with a standard, however complicated and missing scientific features sqlite -- Designed for relat…
I've seen ubiquitous use of tab-separated value files instead of csv, as as simpler format without quoting support and a restriction that your data fields can't contain tabs or newlines, which (unlike commas) is okay for many scenarios.
tab on terminal actually means to "indent to next align point (which is usually multiple of 8)
I literally don't get why JSON is bad: [{row1}, {row2}, {row3}] The fact that it can do more is in no way a negative. Can even make a limited JSON parser with reduced capabilities. And with JSON can do more definitions like header names vs column names vs just arrays of arrays.
From TFA: >> XML and Javascript are tree structures and not suitable for efficiently storing tabular data (plus other issues).
Don't know why csv cant be done in parallel by splitting across lines.