Looks like it also supports SQLite for input, but not for output. That might be a nice addition.
Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
21–30 of 64 posts
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#22Trino can do this as well.
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#23The one thing everyone here is missing so far is that it's a Rust binary, distributed on PyPi. That's brilliant.
I’m all in on using PyPI for binary distribution. Couple that with Python Venv and you have a brilliant system for per project dependancies.
I created this project for distributing Node via PyPI: https://pypi.org/project/nodejs-bin/
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#24This is cool...Totally reminded me about several tools pop up on HN every now and then in the past for similar task so i did a quick search:
clickhouse-local - https://news.ycombinator.com/item?id=22457767
q - https://news.ycombinator.com/item?id=27423276
textql - https://news.ycombinator.com/item?id=16781294
simpql- https://news.ycombinator.com/item?id=25791207
We need a benchmark i think..;)
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#25The one thing everyone here is missing so far is that it's a Rust binary, distributed on PyPi. That's brilliant.
I wrote a bit about this pattern here: https://simonwillison.net/2022/May/23/bundling-binary-tools-...
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#26This is cool...Totally reminded me about several tools pop up on HN every now and then in the past for similar task so i did a quick search: clickhouse-local - https://news.ycombinator.com/item?id=22457767 q - https://news.ycombinator.com/item?id=27423276 textql - https://news.ycombinator.com/item?id=16781294 simpql- https://news.ycombinator.com/item?id=25791207 We need a benchmark i think..;)
Shameless plug. A desktop app: https://superintendent.app
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#27This is cool...Totally reminded me about several tools pop up on HN every now and then in the past for similar task so i did a quick search: clickhouse-local - https://news.ycombinator.com/item?id=22457767 q - https://news.ycombinator.com/item?id=27423276 textql - https://news.ycombinator.com/item?id=16781294 simpql- https://news.ycombinator.com/item?id=25791207 We need a benchmark i think..;)
Shameless plug. A desktop app: https://superintendent.app
we need more shameless plug on this thread ... the more benchmark, the better! I think this would be a fun weekend afternoon :)
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#28As I commented on a recent similar discussion, these tools can't be used for update or insert. As useful as querying might be, it's terribly misleading to claim to "run SQL" if you can't change the data, since that's such a critical part of an SQL database.
i disagree
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#29As I commented on a recent similar discussion, these tools can't be used for update or insert. As useful as querying might be, it's terribly misleading to claim to "run SQL" if you can't change the data, since that's such a critical part of an SQL database.
90% of SQL usage, or more, is select in slowly changing data contexts.
Maybe in your database. Do you have any validation of that claim in a larger context?
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#30AWS Athena offers something similar. You can build tables off of structured text files (like log files) in S3 and run SQL queries.
What’s the performance like though?
I have not put it through any stress tests. I look at this kind of tool as a nice convenience. If I needed something high throughput I'd probably want a full baked data warehouse pipeline.