Live data from Hacker News

Q – Execute SQL on text. Supports joins across files, RPM available

github.com

41–46 of 46 posts

Re: Q – Execute SQL on text. Supports joins across files, RPM available

#41

Eh, q as a name is going to be a serious problem as q is the language for programming kdb, a column-oriented database. As both are related to databases, it's difficult to defend this name.

That was my initial impression, also... I though kx had added text-processing to kdb.

Re: Q – Execute SQL on text. Supports joins across files, RPM available

#42
I'm still not seeing anything easier than ETLing into a regular old database, at which point you have the power and flexibility of a regular old database. Which is pretty spiffy, actually.

Are people really so bad at databases that they'll gladly suffer hacks like this to avoid using one?

Re: Q – Execute SQL on text. Supports joins across files, RPM available

#43
post #9

If you need only a subset of full SQL, e.g. just joins, counting/aggregation, and date manipulation (as in several of these examples), I've found it fairly easy to work with a mixture of the standard Unix join(1) [found on nearly all systems], and some of the additions from Google's crush-tools ( https://code.google.com/p/crush-tools/ ), mainly 'aggregate', 'grepfield', 'funiq', and 'convdate'. I find chaining them t…

Another interesting alternative could be using or writing a PostgreSQL foreign data wrapper.

There is one that supports tabular data (file_fdw), and another one for JSON files (json_fdw). If you have files in other formats, you can also write your fdw for it. This way, you get complete SQL coverage.

Also, if you don't want to pay the overhead of parsing the file every time, you can use the new materialized feature for caching: http://www.postgresql.org/docs/9.3/static/rules-materialized...

(Disclaimer: Enthused Postgres user.)

Re: Q – Execute SQL on text. Supports joins across files, RPM available

#44

What I really wished for when I read the title was for something that could make me write regexes but verbosely.

You can write more verbose regex using Named Capture Buffers. Here is an example I posted on HN not long ago: https://news.ycombinator.com/item?id=6895126

NB. Follow link to original post to compare against standard regex version.

There are also some nice grammar parsers available in some languages which make this even easier. For examples of this see Perl6 Rules/Grammar, Perl5 Regexp::Grammars or (for something which doesn't used regex at all is) Rebol Parse.

For eg. Here is my Rebol version of the HN post above: http://www.reddit.com/r/programming/comments/1smpa1/why_rebo...

And here is a presentation which shows a great example using Perl6 grammars: http://jnthn.net/papers/2014-fosdem-perl6-today.pdf

Refs:

- http://en.wikibooks.org/wiki/Perl_6_Programming/Grammars

- http://en.wikipedia.org/wiki/Perl_6_rules

- https://metacpan.org/pod/Regexp::Grammars

- http://www.rebol.com/docs/core23/rebolcore-15.html

- http://blog.hostilefork.com/why-rebol-red-parse-cool/

PS. Alternatively f you looking for something interactive then checkout tools like these: http://rebol.informe.com/blog/2013/07/01/parse-aid/ | https://metacpan.org/pod/Regexp::Debugger

Re: Q – Execute SQL on text. Supports joins across files, RPM available

#45
post #35

Earlier quoted context omitted.

Haha, well fair enough - even more reason to have checked I suppose. Maybe textQ would be more appropriate given this project's use case?

Funny enough, another commenter mentioned a similar project in Go called "TextQL." https://news.ycombinator.com/item?id=7290739 https://github.com/dinedal/textql

This is why I should not be allowed to name things, clearly.

Re: Q – Execute SQL on text. Supports joins across files, RPM available

#46
post #35

Earlier quoted context omitted.

Funny enough, another commenter mentioned a similar project in Go called "TextQL." https://news.ycombinator.com/item?id=7290739 https://github.com/dinedal/textql

This is why I should not be allowed to name things, clearly.

Nah, I don't see a problem with name collisions in small or relatively unrelated projects like this.
Post reply on HN