Earlier quoted context omitted.
The football data looks easy for a human to read but a pain in the arse for a program to consume. Personally I think it's terrible, and the fact that they have had to develop a custom 'sportsdb' tool to manage it rather than using something generic like 'jq' is telling. https://github.com/openfootball/england/blob/master/2019-20/... To properly parse this file you need to write a parser that cut fixed-width fields (w…
> rather than using something generic like 'jq' is telling. The best generic tool for managing (structured) data is SQL. Once you have the datasets imported (via the custom readers / loaders) it's just plain SQL (and works with SQLite, PostgreSQL, MySQL, etc.)
For small to medium-sized datasets, a nice middleground would be SQL dumps. Put the dumps in Git for versioning and diffing, and load them into $DATABASE for actual queries.