Q – Execute SQL on text. Supports joins across files, RPM available

Post by j_s » Mon, Feb 24, 2014, 5:46 PM UTC

unwind wrote: post #2

It's written in Python, and seems to use SQLite under the hood. I guess it just applies tokenization and throws the text into a temporary database. Quite similar to the Go project https://github.com/dinedal/textql , at least superficially.

  > MS ADO / ODBC
  > Perl DBI
  > npm j (with jqa)
  > Ruby (csv2sqlite) 
  > Python (csvkit)
  > Go (textql, comp)
  > Java (optiq, openrefine, H2 SQL)
  > R (sqldf)
  > Haskell (txt-sushi)
  > XML (xmlstarlet, xmllint, xmlstar)
  > HTML (HtmlAgilityPack, Chrome $x())
  > Postgres file_fdw
  > Oracle external tables
  > SQL Server OPENDATASOURCE and OPENQUERY
  > Log file viewers (MS LogParser, Apache asql, lnav)

Post by harelba » Mon, Feb 24, 2014, 6:56 PM UTC

zeckalpha wrote: post #12

Don't join ( https://en.wikipedia.org/wiki/Join_%28Unix%29 ), sed, and grep get you most of the way there?

Post by rpicard » Mon, Feb 24, 2014, 7:15 PM UTC

hipsters_unite wrote: post #20

mjn wrote: post #13

Earlier quoted context omitted.

And the JS promise library in turn collided with the name of Q, the array-processing language, which itself collided with the name of another programming language named Q ( http://q-lang.sourceforge.net ). If you're naming a tech-related thing after about 1980, the single-letter names are all taken...

Haha, well fair enough - even more reason to have checked I suppose. Maybe textQ would be more appropriate given this project's use case?

Post by gtrubetskoy » Mon, Feb 24, 2014, 7:32 PM UTC

Post by peterwwillis » Mon, Feb 24, 2014, 9:40 PM UTC

theOnliest wrote: post #24

peterwwillis wrote: post #14

Earlier quoted context omitted.

I'm probably heavily biased, but to me Perl is the best command-line regex tool. Perl was invented to gather data and report on it, and its regex engine is incredibly fast and powerful. As an added bonus it supports some Python and PCRE-specific extensions. But this Q app is useful for people who either don't know Perl or can get what they need done faster with SQL than with scripting. In terms of 'verbosity' you can…

> As an added bonus it supports some Python and PCRE-specific extensions. This is a bit of a strange thing to say, since nearly all of the advanced regex features showed up in Perl first . PCRE stands for "Perl-compliant regular expressions," so there's certainly no extensions there that didn't originally come from Perl. I'm less sure about Python, but I get the sense that they borrow from Perl regular expressions as…

  PCRE/Python Support
  
  As of Perl 5.10.0, Perl supports several Python/PCRE-specific extensions to the
  regex syntax. While Perl programmers are encouraged to use the Perl-specific
  syntax, the following are also accepted:
  
      (?Ppattern)
      Define a named capture group. Equivalent to (?pattern).
  
      (?P=NAME)
      Backreference to a named capture group. Equivalent to \g{NAME} .
  
      (?P>NAME)
      Subroutine call to a named capture group. Equivalent to (?&NAME).