Isn't doing data science locally in a browser like tieing one arm to your back and trying to tie your shoe laces? Why would you do it?
Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
71–80 of 84 posts
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#72200mb of data is not a large file, and chromium tabs have a memory limit of something ridiculously low so actual large 20-100gb datasets render this useless.
Sometimes you need a scooter, sometimes you need a truck. I think a snappy interface for <1gb datasets is really neat and super useful for certain kinds of data
You can now use the file api to work with terabyte sized files if your disk can handle it.
https://developer.chrome.com/docs/capabilities/web-apis/file...
Browser is pretty powerful nowadays.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#73This is really promising, please look into embedded analytics. We are struggling with building our embedding for SAAS. Now we are doing it with cube and superset. All the alternatives are expensive or hard to implement.Being able to offload the analytics to the browser is a plus.
[dead]
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#74That looks great, I love all these new use cases for WASM that are starting to pop up. Are you planning on supporting parquet files any time soon? I'd love an easy way to just drop a parquet file on there and easily visualize time-series data with it (e.g. sensor readings). Small issue I noticed: When you chart time-series data, the timestamp axis is just represented as an integer rather than a meaningfully formatted…
Thanks for the feedback! > Are you planning on supporting parquet files any time soon? Yes. This is an oversight on our part - duckdb natively supports Parquet and Arrow files but we just haven't gotten around to adding upload support for those yet. It's a small change - landing in main tomorrow! On the bug, yes, we've been trying to parse timestamps but it's just been very finicky. I'll try to fix this! Thanks for t…
I think this could probably replace the functionality provided by datasette?
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#75What is this offering over something like datasette?
Or even OpenRefine?
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#76Wow! Is there any way this could support SQLite databases?
DuckDB supports SQLite through extensions ( https://duckdb.org/docs/extensions/sqlite.html ) So, it should be pretty straightforward to let folks drop a SQLite file instead of a CSV file! I haven't been able to get extensions to work on WASM so far though but I will definitely take a look to see if we can make this work!
Fantastic! Would you like me to make an issue for this?
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#77Earlier quoted context omitted.
Thanks for the feedback! > Are you planning on supporting parquet files any time soon? Yes. This is an oversight on our part - duckdb natively supports Parquet and Arrow files but we just haven't gotten around to adding upload support for those yet. It's a small change - landing in main tomorrow! On the bug, yes, we've been trying to parse timestamps but it's just been very finicky. I'll try to fix this! Thanks for t…
I think this could probably replace the functionality provided by datasette?
Datasette was definitely a huge influence for this and I'm a big of @simonw and his work.
However, there's a couple of major differences:
- Datasette - to me - is a tool for devs - it takes a fair bit of tech know-how to understand how to load new data in datasette, install plugins and deploy an instance. I wanted Pretzel to have a "low-ceiling, high-floor" - meaning the tool should work well for non-technical folks but also shouldn't hinder technical folks and power users
- Datasette has a unique way to explore data based on "facets" where you can dig through data. It's a powerful way to explore but I personally don't think Datasette is the best tool for complex data transforms - its strengths lie in exploring datasets instead.
- For accessibility and speed, we wanted something browser local and fast - hence the focus on DuckDB and canvas rendered tables. Datasette requires setting up an instance and that's additional friction for non-tech folks and isn't particularly designed for large datasets (is my impression - I might be wrong)
- Lastly, we have ton of future plans - including in-browser python support via WASM (Pyodide), local LLM support, ability to connect databases etc.
I think there's space for both tools!
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#78What is this offering over something like datasette? https://datasette.io/ Or even OpenRefine? https://openrefine.org/
Hey, I left a comment in another thread about the differences:
https://news.ycombinator.com/item?id=39732135
Didn't know about OpenRefine - but one major difference is Pretzel is web-based meaning no installation necessary (important for work laptops, for eg). We're also going to be adding a ton of features to this - think Jupyter notebooks crossed with Metabase crossed with Datasette/Pretzel.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#79What is this offering over something like datasette? https://datasette.io/ Or even OpenRefine? https://openrefine.org/
Hey, I left a comment in another thread about the differences: https://news.ycombinator.com/item?id=39732135 Didn't know about OpenRefine - but one major difference is Pretzel is web-based meaning no installation necessary (important for work laptops, for eg). We're also going to be adding a ton of features to this - think Jupyter notebooks crossed with Metabase crossed with Datasette/Pretzel.
Thanks for the reply. I thought I'd made some obvious error when I asked and was getting downvoted without comment. I appreciate you time in replying to what may have been a stupid question.
Re: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL
#80I think no one really enjoys working with sheet or excel. Apart from the inability to handle large data, have such a high/unintuitive learning curve for most data analysis operations. I can see this tool become a ‘metabase’ for everyday usage of data analysis.
Thanks! Yes, we're going to build the make modern data tool with a low barrier to entry (so non-devs, non-data folks can use it) but that also works great for power users (a way better jupyter notebook, with modern IDE like capabilities - pair programming, copilot, extension, out-of-the-box linting, testing etc).
It's almost a meta thing but I think dev-tools have been getting better and better over the last 10-20 years while data tools seem to be stuck in the past. Devs, when then see friction, go build something to solve it while this tendency is much less common in data scientists, for eg.