Live data from Hacker News

Show HN: FSQL – Search through your file system with SQL-esque queries

github.com

11–20 of 132 posts

Re: Show HN: FSQL – Search through your file system with SQL-esque queries

#12
Nice! I'm actually working on a similar project to push lsof and files from /proc into some postgres tables. Lets me do cool things like query log files across a ~6000 server infrastructure similar to:

  SELECT distinct(l.name) 
    FROM lsof l, lsofer_runs r 
  WHERE l.lsofer_id = r.id 
    AND fd_type = 'REG' 
    AND l.fd ~ '[0-9][uw]' 
    AND l.name like '%log' 
  GROUP BY l.name, r.hostname 
  ORDER BY name
Best of luck!

Re: Show HN: FSQL – Search through your file system with SQL-esque queries

#13
post #12

Nice! I'm actually working on a similar project to push lsof and files from /proc into some postgres tables. Lets me do cool things like query log files across a ~6000 server infrastructure similar to: SELECT distinct(l.name) FROM lsof l, lsofer_runs r WHERE l.lsofer_id = r.id AND fd_type = 'REG' AND l.fd ~ '[0-9][uw]' AND l.name like '%log' GROUP BY l.name, r.hostname ORDER BY name Best of luck!

so you're rewriting osquery? https://osquery.io/

Re: Show HN: FSQL – Search through your file system with SQL-esque queries

#15

Nice project, wish you the best ! Although tbh, I personally won't use this simply because I know enough of find(1) to not see the cognitive overhead of switching to sql to do filesystem /queries/. Any examples where this would be better than using find (with the occasional filter thrown in) ?

Subselects would be a pretty awesome feature.

"select name from foo where name not in (select name from ../bar where date I'm usually fine with `find`, but when doing things more interesting than just "find files in this directory that are not in that directory", while uncommon, tend to make me think about my pipeline a bit.

Re: Show HN: FSQL – Search through your file system with SQL-esque queries

#17
post #12

Nice! I'm actually working on a similar project to push lsof and files from /proc into some postgres tables. Lets me do cool things like query log files across a ~6000 server infrastructure similar to: SELECT distinct(l.name) FROM lsof l, lsofer_runs r WHERE l.lsofer_id = r.id AND fd_type = 'REG' AND l.fd ~ '[0-9][uw]' AND l.name like '%log' GROUP BY l.name, r.hostname ORDER BY name Best of luck!

so you're rewriting osquery? https://osquery.io/

Hah, apparently.

Re: Show HN: FSQL – Search through your file system with SQL-esque queries

#18
post #12

Nice! I'm actually working on a similar project to push lsof and files from /proc into some postgres tables. Lets me do cool things like query log files across a ~6000 server infrastructure similar to: SELECT distinct(l.name) FROM lsof l, lsofer_runs r WHERE l.lsofer_id = r.id AND fd_type = 'REG' AND l.fd ~ '[0-9][uw]' AND l.name like '%log' GROUP BY l.name, r.hostname ORDER BY name Best of luck!

Looks very cool, is the code available anywhere? Would love to take a look.

Re: Show HN: FSQL – Search through your file system with SQL-esque queries

#19
post #15

Nice project, wish you the best ! Although tbh, I personally won't use this simply because I know enough of find(1) to not see the cognitive overhead of switching to sql to do filesystem /queries/. Any examples where this would be better than using find (with the occasional filter thrown in) ?

Subselects would be a pretty awesome feature. "select name from foo where name not in (select name from ../bar where date I'm usually fine with `find`, but when doing things more interesting than just "find files in this directory that are not in that directory", while uncommon, tend to make me think about my pipeline a bit.

Out of curiosity, I'm interested in how folks would do the "in this not that" folder query. At a gut shot, I'd assume that diff would be used. I'm about to dig through the find man page to see if it has something directly to help.

Re: Show HN: FSQL – Search through your file system with SQL-esque queries

#20
post #8

Reminds me of osquery [0]. [0] - https://github.com/facebook/osquery

Thanks for sharing! I've installed this locally and I'm really impressed by how easy to use and powerful this is! I must have missed the previous mentions on HN [0],[1].

[0] https://news.ycombinator.com/item?id=8528460 [1] https://news.ycombinator.com/item?id=12600790

Post reply on HN