The one thing everyone here is missing so far is that it's a Rust binary, distributed on PyPi. That's brilliant.
Can you explain the advantages of this vs cargo?
cargo is not a binary distribution.
Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
11–20 of 64 posts
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#12As I commented on a recent similar discussion, these tools can't be used for update or insert. As useful as querying might be, it's terribly misleading to claim to "run SQL" if you can't change the data, since that's such a critical part of an SQL database.
Question: I've built something that supports full CRUD, and queries that span multiple data sources with optimization and pushdown
What kind of headline would make you want to read/try such a thing?
(I'm planning on announcing it + releasing code on HN but have never done so before)
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#131) roapi is built with some wicked cool tech
2) the author once answered some questions I posted on Datafusion, so they're cool in my book
Here are my anecdotes.
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#14As I commented on a recent similar discussion, these tools can't be used for update or insert. As useful as querying might be, it's terribly misleading to claim to "run SQL" if you can't change the data, since that's such a critical part of an SQL database.
Question: I've built something that supports full CRUD, and queries that span multiple data sources with optimization and pushdown What kind of headline would make you want to read/try such a thing? (I'm planning on announcing it + releasing code on HN but have never done so before)
Show HN: Read and update Arrow, Parquet and xxxx files using SQL
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#15It is pretty cool. py-spy has also been doing this for a few years
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#16Bye bye jq and your awful query syntax.
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#17The one thing everyone here is missing so far is that it's a Rust binary, distributed on PyPi. That's brilliant.
You can get a statically-linked binary release from GitHub which depends on nothing (I think).
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#18AWS Athena offers something similar. You can build tables off of structured text files (like log files) in S3 and run SQL queries.
What’s the performance like though?
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#19As I commented on a recent similar discussion, these tools can't be used for update or insert. As useful as querying might be, it's terribly misleading to claim to "run SQL" if you can't change the data, since that's such a critical part of an SQL database.
90% of SQL usage, or more, is select in slowly changing data contexts.
Re: Run SQL on CSV, Parquet, JSON, Arrow, Unix Pipes and Google Sheet
#20Earlier quoted context omitted.
Question: I've built something that supports full CRUD, and queries that span multiple data sources with optimization and pushdown What kind of headline would make you want to read/try such a thing? (I'm planning on announcing it + releasing code on HN but have never done so before)
Show HN: Read and update Arrow, Parquet and xxxx files using SQL
It works on databases and arbitrary data sources too though