Can we bookmark or localstorage our etl?
Have a look at https://sql-workbench.com, it supports query sharing via URL, as well as sharing visualizations. Let me know if you have questions!
Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
31–40 of 84 posts
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#32I was recently searching for something like this - the ability to write PRQL queries on mobile is my ideal usecase. As you note, PRQL is perfect for putting together queries with drop-downs, so I'm a huge fan of the UI that you've put together. Feature request: Exposing the PRQL would be great, and using tabs to switch between the query and results would make it possible to work on mobile. Lastly, bookmarking those q…
Yes! I'm heartened to see people realize the the benefits of PRQL and chained transforms over vanilla SQL (with some caveats!). We'll definitely add a PRQL/SQL block to the UI soon (you will be able to toggle to select which one you'd like). On that point - PRQL doesn't natively support PIVOT statements. As a hack, we made our own flavor of PRQL with a PIVOT statement and we parse that to SQL CTEs to make it work for…
[PRQL dev here]
Love what you've done and thanks for building on PRQL!
We're very happy to add something like a PIVOT statement. In the long term, we'd have to think about how to make it work deeply in PRQL, since the column names are suddenly runtime dependent. In the short term, we should definitely make PRQL work for your case — it's important that there's an escape hatch for things that aren't natively supported by PRQL yet. (We have s-strings but I'm guessing they don't cover this specific case?)
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#33The DuckDB WASM space is really heating up! I released https://sql-workbench.com a few weeks ago, which can be used to query and visualize Parquet, CSV, JSON and Arrow data. There’s also a accompanying tutorial blog post at https://tobilg.com/using-duckdb-wasm-for-in-browser-data-eng...
I just like to thank you (again) for how much you've made a lot of what is required to get duckdb up and started (esp on AWS) much easier.
Interesting to watch the similarities (duckdb wasm) and differences (python vs js) between your SQL IDEs in the browser. Exciting stuff.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#34The DuckDB WASM space is really heating up! I released https://sql-workbench.com a few weeks ago, which can be used to query and visualize Parquet, CSV, JSON and Arrow data. There’s also a accompanying tutorial blog post at https://tobilg.com/using-duckdb-wasm-for-in-browser-data-eng...
I just like to thank you (again) for how much you've made a lot of what is required to get duckdb up and started (esp on AWS) much easier. Interesting to watch the similarities (duckdb wasm) and differences (python vs js) between your SQL IDEs in the browser. Exciting stuff.
Thank you! It’s really great that the in-browser data analysis ecosystem is really firing up… Exited to see what others build
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#35200mb of data is not a large file, and chromium tabs have a memory limit of something ridiculously low so actual large 20-100gb datasets render this useless.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#36This is really promising, please look into embedded analytics. We are struggling with building our embedding for SAAS. Now we are doing it with cube and superset. All the alternatives are expensive or hard to implement.Being able to offload the analytics to the browser is a plus.
Thanks! This is interesting - just to make sure I'm understanding you correctly - you're trying to build analytics into your SaaS tool to offer it to your users, correct? And you're using Superset to build the analytics inside your SaaS app?
I'll definitely look into it. For the moment, I know a team trying to solve this exact problem (they're not open-source/free but I think they're pretty inexpensive) - happy to connect you with them if you like :)
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#37200mb of data is not a large file, and chromium tabs have a memory limit of something ridiculously low so actual large 20-100gb datasets render this useless.
Sometimes you need a scooter, sometimes you need a truck.
I think a snappy interface for <1gb datasets is really neat and super useful for certain kinds of data
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#38200mb of data is not a large file, and chromium tabs have a memory limit of something ridiculously low so actual large 20-100gb datasets render this useless.
I looked up the limit and as of 2021, tabs seem to have been limited to 16GB which is moderate in size for an in-memory dataset. However, I know WASM has a hard limit of 4GB without Memory64. Data size is all relative.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#39200mb of data is not a large file, and chromium tabs have a memory limit of something ridiculously low so actual large 20-100gb datasets render this useless.
This is certainly true - I'm not saying "large file" in the colloquial sense of the "big data" but rather as in - a file you might want to open in Excel/Google Sheets. I've worked actual large datasets before - upwards of 500GB - pretty often before an I really wouldn't think about using my laptop for a such a thing!
We are thinking of making data connectors to major DBs though so you should be able to do a similar style visual analysis while keeping the compute on your DB.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#40I enjoyed your demo video and played around with a CSV of my own. This is a cool project, but I have a (perhaps stupid) question:
Why should I use this tool over Excel or Google Sheets?
I don't mean this question to be a challenge. Just trying to understand which sorts of tasks would I be better off using Pretzel as opposed to a traditional spreadsheet tool.