Live data from Hacker News

TextQL: Execute SQL Against CSV or TSV

github.com

91–95 of 95 posts

Re: TextQL: Execute SQL Against CSV or TSV

#91
post #5

In SQLite you can just do sqlite> .import myfile.csv mytable sqlite> select ... What does this tool give you that SQLite doesn't do out of the box?

You skipped the line to create the bare table first in SQLite: https://www.sqlite.org/cvstrac/wiki?p=ImportingFiles

Link is to a page that says obsolete since 2009. I wasn't able to find similar page in the docs, can you help?

Re: TextQL: Execute SQL Against CSV or TSV

#92
post #10

If you work with CSV a lot, then also check out xsv, "a command line program for indexing, slicing, analyzing, splitting and joining CSV files", from Andrew Gallant (burntsushi of Go and Rust fame): https://github.com/BurntSushi/xsv

`xsv` looks great, I will try it out.

My main hesitation: Its another DSL to learn, there is a big benefit to keeping this kind of work within the SQL world for me (or R, python, etc depending on the user).

On the other hand, getting table stats and frequencies right from the shell is a huge time saver over SQL.

Re: TextQL: Execute SQL Against CSV or TSV

#93

Earlier quoted context omitted.

You skipped the line to create the bare table first in SQLite: https://www.sqlite.org/cvstrac/wiki?p=ImportingFiles

Link is to a page that says obsolete since 2009. I wasn't able to find similar page in the docs, can you help?

Help with what ? If you want to import a csv file into SQLite, just follow the "obsolete" docs which probably still work. If you want help finding a similar page in the docs, just read all the docs or download them and grep. It may or may not exist.

Re: TextQL: Execute SQL Against CSV or TSV

#94
Yawn. We've had this in Perl for decades using the DBI Shell (dibsh). DBI is a common database interface. To connect to a data source, one uses an appropriate driver (by convention, in the DBD name space, such as DBD::Pg or DBD::MySQL) to get a db handle that can run SQL commands. There are a lot of drivers for things like CSV files, in memory database, etc.

Plus you can access all this programmatically and not just through the dbish command. Since this is a central part of Perl infrastructure, there are a ton of tools that extend the DBI, from a variety of ORMs like DBIx::Class, to data export and even DDL delta management.

Generalized Database Interface: https://metacpan.org/pod/DBI DBI Shell program: https://metacpan.org/pod/DBI::Shell CSV driver: https://metacpan.org/pod/DBD::CSV

Re: TextQL: Execute SQL Against CSV or TSV

#95

Does this have any benefits over lnav[1]? [1] https://www.lnav.org -- a feature-rich, powerful, flexible "mini-ETL" with an embedded sqlite engine -- is one of my all-time favorite CLI tools that somehow remains under the radar

http://innolitics.com/10x/graduate-from-sed-and-sort-to-lnav...
Post reply on HN