Q – Execute SQL on text. Supports joins across files, RPM available
1–10 of 46 posts
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#2It's written in Python, and seems to use SQLite under the hood.
I guess it just applies tokenization and throws the text into a temporary database.
Quite similar to the Go project https://github.com/dinedal/textql, at least superficially.
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#3Cool idea!
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#4Dataset (https://dataset.readthedocs.org/) is a similar project that provides ad-hoc SQL querying of structured data - example usage:
import dataset
db = dataset.connect('sqlite:///:memory:')
table = db['sometable']
table.insert(dict(name='John Doe', age=37))
table.insert(dict(name='Jane Doe', age=34, gender='female'))
john = table.find_one(name='John Doe')Re: Q – Execute SQL on text. Supports joins across files, RPM available
#5Name collision with Q/kdb+.
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#6This seems like a cool project, but Q is already a well-used JS promise library so there's a bit of a name conflict there even if the libs are aimed at very different tasks...
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#7What I really wished for when I read the title was for something that could make me write regexes but verbosely.
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#8This seems like a cool project, but Q is already a well-used JS promise library so there's a bit of a name conflict there even if the libs are aimed at very different tasks...
Any library with a really short name (<= 2 letters) should be prepared for name collisions. If they wanted to avoid name collisions, they should have chosen a slightly longer, more descriptive name in the first place. I think in this context (i.e. really short name) it doesn't really matter who was first.
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#9If you need only a subset of full SQL, e.g. just joins, counting/aggregation, and date manipulation (as in several of these examples), I've found it fairly easy to work with a mixture of the standard Unix join(1) [found on nearly all systems], and some of the additions from Google's crush-tools (https://code.google.com/p/crush-tools/), mainly 'aggregate', 'grepfield', 'funiq', and 'convdate'. I find chaining them together a bit easier than writing monolithic SQL statements, but there's probably some crossover point at which that wouldn't be true.
It'd be interesting to compare runtimes as well. I would guess that there's some overhead in loading into the DB up front, but that you might gain some speedup by converting longer chains of Unix pipes into one SQL query. On the other hand you might lose some parallelism. Would take some testing on different kinds of queries and data sets to get an idea of the pros/cons I think.
Re: Q – Execute SQL on text. Supports joins across files, RPM available
#10I have ended up with MySQL and PostgreSQL on nearly every linux I've installed. It is baffling to me that someone would decline these tools in favor of text file drivers or SQLite, with the exception of using SQLite as an embedded config database or something.