51–60 of 127 posts
RDF, RDF, RDF, RDF, RDF, RDF, RDF. Yes. Please use RDF. And please use a common vocabulary. If you want to know how to do this? Have a look at the UK governments open data portal (which uses RDF): data.gov.uk And their SPARQL (query) endpoint: http://data.gov.uk/sparql
http://en.wikipedia.org/wiki/Resource_Description_Framework
I'm interested in knowing a little more about why you like it so much.
My preference is .csv, because it can be read by almost any program. Except the one most used for it: Excel. Excel sucks at csv, especially if the recipients have various internationalized versions. It can't deal properly with multi-line strings and spaces, but especially the internationalization is hell (semi-colons instead of commas, decimal separators, date formats). Many times have I dealt with CSV-exports that c…
You left out the absolute worst bit: Excel tries to "helpfully" guess the correct data format for each cell in a CSV file. This can lead to silent data loss. Long strings of numbers are converted to a Numeric type that only stores a certain number of significant figures. The string "1-5" is converted to a Date field with the value "5-Jan"
As long as the PDF contains text (and isn't just scans of paper), it's usually not TOO hard to select and copy the relevant tables/text, paste into a text editor, and use regex's to transform it into whatever you want (CSV, SQL insert statements, etc.) Since there's no universal data format (e.g. the data equivalent of PDF), I pretty much assume that whatever format data comes in, I'll have to be doing regex transfor…
Earlier quoted context omitted.
You left out the absolute worst bit: Excel tries to "helpfully" guess the correct data format for each cell in a CSV file. This can lead to silent data loss. Long strings of numbers are converted to a Numeric type that only stores a certain number of significant figures. The string "1-5" is converted to a Date field with the value "5-Jan"
If you are generating the CSV yourself, save yourself some agony and just wrap the text in ="" $ cat test.csv ="12.34567890124312341234123412341234",="1-5" $ open -a Microsoft\ Excel test.csv
JUST TREAT THE NUMBERS LIKE TEXT YOU STUPID PROGRAM.
000001 === 000001 != 1
My preference is .csv, because it can be read by almost any program. Except the one most used for it: Excel. Excel sucks at csv, especially if the recipients have various internationalized versions. It can't deal properly with multi-line strings and spaces, but especially the internationalization is hell (semi-colons instead of commas, decimal separators, date formats). Many times have I dealt with CSV-exports that c…
I've always found the C in CSVs to be needlessly limiting. I'm not aware of any tool that contextualise commas in addresses and other "free text" fields that might form part of the data dump/extract. For this reason, I am strongly in favour of pipe-separated values in such files. The probability that a data extract has a pipe symbol in any data field is quite small (in my experience, it's been 0 so far). The first th…
TOML will solve all these problems. More structured than JSON, easier to consume than XML. https://github.com/mojombo/toml
Earlier quoted context omitted.
From someone currently dealing with Excel's CSV writing for a business application, I heartily concur. There is a decade old bug (at least) where the CSV save as format does not properly handle Unicode, and there's no simple way to handle it down the pipeline. I guess the Office team would rather work on the metro ribbon.
Well CSV has no real specification, or way to store metadata eg about encoding. It is a terrible format.
---
People can be great at using their particular tool, and then they'll do something to make you realise that they're just an expert with that tool and that their wider computer knowledge isn't so great.
I've had people want online bank statements as PDFs not CSVs because "anyone can change the Excel file"; I had people asking me to scan engineering drawing on paper to gif files ("Nice and small! We don't have enough space for that other format!" (we did have space)) and then import those gifs into AutoCAD so we could update our versions of the drawings when the customer updated their versions. (The guy before me had given up trying to explain why it was a dumb idea and had just scanned about 1,000 (of about 10,000) documents and saved them as gif.)
Some people aren't particularly sharing data. They're just sharing their information that they've got from that data; the interpretation they're making.