Neat. I did a similar thing for analysing the output of commands that can produce JSON (SLURM in this case). Worked pretty well but I immediately ran into PRQL's lack of support for DuckDB's struct types.
Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
11–20 of 84 posts
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#12this is great, thanks for sharing I'm currently using a self hosted instance of lightdash connected to a dbt project and I can see this being really efficient for data exploration for business users quite interesting!
Thank you. We're definitely looking to get to a point where you don't need to jump to a jupyter notebook for simple analyses, especially things which are hard to do in SQL (a basic linear regression model for eg).
Happy to chat at prasoon [at] withpretzel [dot] com if you need any integration help!
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#13Nice to see DuckDB-Wasm get some love Curious from the pure performance point of view whether you have performed any benchmarking against any of the alternatives for this sort of in-browser data work?
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#14Just tried it out - thanks for sharing.
We're a sales tech startup and I've had to look through large CSVs with prospecting information in the past - anything over 10 MB crashes my browser. I use a Mac so I don't have Excel. This looks great for simple data manipulations. I tried out a large CSV I had, and it loaded without a problem.
Quick bug report: The filter interface seems to be slow for me though with the large file. Also, one feature that would be really helpful would be connecting this to a database and also some way to share my workflow/analysis.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#15Neat. I did a similar thing for analysing the output of commands that can produce JSON (SLURM in this case). Worked pretty well but I immediately ran into PRQL's lack of support for DuckDB's struct types.
Yes, there's definitely some downsides (which honestly, we didn't realize when we started to build this). For us, it was the lack of a PIVOT statement so I had to make a modified dialect of PRQL that supports PIVOT, then split on the final PRQL into PIVOT and non-PIVOT chunks, and then convert it to chained SQL CTEs. Annoying work for sure.
As a result, I'll be adding support for PIVOTs very soon, either in the main repo or just in a temporary fork.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#16Nice to see DuckDB-Wasm get some love Curious from the pure performance point of view whether you have performed any benchmarking against any of the alternatives for this sort of in-browser data work?
That's an interesting point! No we haven't really - for now, I asked a bunch of friends to send me large CSVs - I loaded them on Google Sheets to see how it performs (usually terribly - crashes my browser tab or it's unbearably slow). Then I try the CSV in Pretzel with some filters and pivots and if I don't feel any annoyance with speed, it's a success in my book for now.
We'll definitely need to figure out (a) what's the right thing to measure, and (b) an automated testing/CI to check for regressions and do perf testing
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#17Just tried it out - thanks for sharing. We're a sales tech startup and I've had to look through large CSVs with prospecting information in the past - anything over 10 MB crashes my browser. I use a Mac so I don't have Excel. This looks great for simple data manipulations. I tried out a large CSV I had, and it loaded without a problem. Quick bug report: The filter interface seems to be slow for me though with the larg…
[dead]
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#18Just tried it out - thanks for sharing. We're a sales tech startup and I've had to look through large CSVs with prospecting information in the past - anything over 10 MB crashes my browser. I use a Mac so I don't have Excel. This looks great for simple data manipulations. I tried out a large CSV I had, and it loaded without a problem. Quick bug report: The filter interface seems to be slow for me though with the larg…
Hey Marco, I'll take a look - filters theoretically should be fast, when you create a new filter, it simply reads does a `select * from table limit 1` to get column names
I wasn't sure whether you could query DBs directly from the browser but looks like you can! (https://github.com/alexanderguy/pgress) - will add it to roadmap!
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#19Oh man this is cool. I don't really have much feedback tbh I just think this awesome. I can see myself using this as a handy little utility when data sets get a bit too big for Google Sheets. In terms of it being a product, things I can think of that would be useful to me. - Importing/exporting data from Google Sheets/Google Drive. - Scheduling queries to run and export etc. Not sure if that really aligns to this spe…
That's great feedback, thanks! This tool definitely comes from a place of personal need - beyond just handling large files, I've also never really gelled well with the Excel/Google Sheet model of changing data in place as if you were editing text. I'm a Data Scientist and always preferred the chained data transforms you see in things like dplyr ( https://dplyr.tidyverse.org/ ) or Polars ( https://pola.rs/ ) and I fee…
How ary you
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#20Can we bookmark or localstorage our etl?