21–24 of 24 posts
This is an overly complicated data container format for not much advantage. To be honest, everything you can do with this can be done at the same level or better with SQLite, an actual database system. Having to implement 4 different parers and validation functions spanning a mix of csv, xml and json just to access what is essentially a csv file is not feasible.
This is an overly complicated data container format for not much advantage. To be honest, everything you can do with this can be done at the same level or better with SQLite, an actual database system. Having to implement 4 different parers and validation functions spanning a mix of csv, xml and json just to access what is essentially a csv file is not feasible.
I disagree that there is "not much advantage" in the format though. I use much of the "resources" area of the data container format and find it tremendously helpful for validating the expected datatypes (remember, SQLite has no true datatypes for columns), defining expected values, and defining some of the "ETL" functionality in ETLyte, like derived columns.
Also on the horizon is a fuzzing tool I'm creating to help exercise the boundaries and variations of data that an ETL process can expect, and this wouldn't be possible without a data container format. So again, I think there are very good use cases for it that we haven't even tapped into yet.
Earlier quoted context omitted.
Hi, (I work on the Frictionless Data specifications and tooling at Open Knowledge International.) CSV has many, many warts. However, it is the best thing we have right now for serialising data in a way that is easily read by humans (and consumer-grade software) and machines. Libraries like our Tabulator [1] which is used under-the-hood help provide an API to deal with many of the gotcha's when dealing with the format…
Thanks, will have a look at tabulator. I appreciate a list of validators you published on FD site, can help at least a bit when working with non-techies to vet their data before submission.