If PDF is all you have, at least there is pdfescape:
How to share your data effectively
111–120 of 127 posts
Re: How to share your data effectively
#112'Send me your data in PDF and ONLY PDF' said most courts in the US.
Re: How to share your data effectively
#113Earlier quoted context omitted.
http://tools.ietf.org/html/rfc4180
As I said, no real specification. It reads "This section documents the format that seems to be followed by most implementations". Another part reads "Each field may or may not be enclosed in double quotes (however some programs, such as Microsoft Excel, do not use double quotes at all)." The RFC is a useful collection of anecdotes...
The HTTP 1.0 RFC says the same thing:
http://tools.ietf.org/html/rfc1945#page-4
"This specification describes the features that seem to be consistently implemented in most HTTP/1.0 clients and servers."
Is HTTP not a real specification either?
Re: How to share your data effectively
#114My preference is .csv, because it can be read by almost any program. Except the one most used for it: Excel. Excel sucks at csv, especially if the recipients have various internationalized versions. It can't deal properly with multi-line strings and spaces, but especially the internationalization is hell (semi-colons instead of commas, decimal separators, date formats). Many times have I dealt with CSV-exports that c…
I've always found the C in CSVs to be needlessly limiting. I'm not aware of any tool that contextualise commas in addresses and other "free text" fields that might form part of the data dump/extract. For this reason, I am strongly in favour of pipe-separated values in such files. The probability that a data extract has a pipe symbol in any data field is quite small (in my experience, it's been 0 so far). The first th…
I also find the C in CSVs to be needlessly limiting. Especially since ASCII already comes defined with characters for record separator (30), unit separator(31), and group separator(29).
Re: How to share your data effectively
#115A new tool for pulling tables out of PDFs was released recently by Mozilla Open News http://source.mozillaopennews.org/en-US/articles/introducing...
[deleted]
Re: How to share your data effectively
#116...and horrors of math equations and columns in PDF. We should really come up with some light format where style is separated from data. Make a universal converter from doc and xls files and save world from data stuck in PDFs.
We could call it .. LaTeX.
Why people share exported PDF's not their LaTeX files?
Re: How to share your data effectively
#117Earlier quoted context omitted.
You left out the absolute worst bit: Excel tries to "helpfully" guess the correct data format for each cell in a CSV file. This can lead to silent data loss. Long strings of numbers are converted to a Numeric type that only stores a certain number of significant figures. The string "1-5" is converted to a Date field with the value "5-Jan"
You can specify the field types when opening the file.
Except that when users click on or download a csv-file, it Excel opens it automatically, bypassing any options and not even bothering to check if the contents of the csv file matches its defaults.
This is 90% of use cases with regular users.
For the other 10%, the interface for specifying the field types sucks, doesn't cover certain options and you can't set your preferences.
Imagine being a user and having to do that every day in order to import your daily whatever report, and having to start all over if you make one little mistake.
And that's just for the options Excel supports. Don't even get me started on all the other quirks. Ever tried to read a csv file who's first line starts with "ID"?
Re: How to share your data effectively
#118There is lots of bad advice in this article. OK maybe the article is shorter than I thought. Here, this is bad: Use short variable names with no whitespace. Underscores are usually a safe bet, so instead of "Number of new tuberculosis cases" use "incident_tb" While it might be clear to the domain experts, I had no idea that "incident_tb" meant "Number of new tuberculosis cases" other than that it was placed next to t…
I think the rest of the points are good suggestions but "use short variable names" is a terrible suggestion. "incident_tb" is one of the worst possible names that could be used while still being relevant to the data represented.
It's 2013, software can handle titles longer than 8 characters. Even something like "n_new_tb_incidents" (which is shorter than I'd like) buys a lot more clarity at the expense of a few characters. I'd take that bargain any day.
Re: How to share your data effectively
#119RDF, RDF, RDF, RDF, RDF, RDF, RDF. Yes. Please use RDF. And please use a common vocabulary. If you want to know how to do this? Have a look at the UK governments open data portal (which uses RDF): data.gov.uk And their SPARQL (query) endpoint: http://data.gov.uk/sparql
I much prefer the RDF graph data model over the document-centric tree model of JSON or XML for representing data. It's too bad the need for RDF to be a solid formal knowledge representation format trumped more practical concerns (like being able to represent an ordered list in a simple manner).
Re: How to share your data effectively
#120Earlier quoted context omitted.
As I said, no real specification. It reads "This section documents the format that seems to be followed by most implementations". Another part reads "Each field may or may not be enclosed in double quotes (however some programs, such as Microsoft Excel, do not use double quotes at all)." The RFC is a useful collection of anecdotes...
The HTTP 1.0 RFC says the same thing: http://tools.ietf.org/html/rfc1945#page-4 "This specification describes the features that seem to be consistently implemented in most HTTP/1.0 clients and servers." Is HTTP not a real specification either?
And then laid down which ones would be used and exactly how, and how to specify that you conform. Http is just describing process of how the spec came about. Because there is no metadata in csv you cannot even say how or if you conform.