Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser

hyperparam.app

Post reply on HN
Print view View original

1–10 of 23 posts

Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser

#1

Post by platypii »

For the last year I’ve been developing Hyperparam — a collection of small, fast, dependency-free open-source libraries designed for data scientists and ML engineers to actually look at their data.

- Hyparquet: Read any Parquet file in browser/node.js

- Icebird: Explore Iceberg tables without needing Spark/Presto

- HighTable: Virtual scrolling of millions of rows

- Hyparquet-Writer: Export Parquet easily from JS

- Hyllama: Read llama.cpp .gguf LLM metadata efficiently

CLI for viewing local files: npx hyperparam dataset.parquet

Example dataset on Hugging Face Space: https://huggingface.co/spaces/hyperparam/hyperparam?url=http...

No cloud uploads. No backend servers. A better way to build frontend data applications.

GitHub: https://github.com/hyparam Feedback and PRs welcome!

Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser
hyperparam.app

Re: Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser

#3

Post by platypii »

doppenhe wrote: post #2

Very cool, does `npx hyperparam dataset.parquet` phone home?

Zero telemetry, fully local. It spawns `http-server` on port 2048 and opens your browser at `localhost`. Similar pattern as Jupyter Notebooks. Feel free to audit the code... the server is <200 LOC.

Re: Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser

#5

Post by platypii »

wbradmoore wrote: post #4

Why not WASM? Seems like something like duckdb-wasm or datafusion-wasm can do the same thing?

Duckdb and datafusion are super cool! But they are VERY large wasm blobs (30-40mb each). This is often larger than the data you’re trying to load. And they add complexity with serving and deploying wasm files.

Hyparquet is 10kb of pure js, and so its trivial to deploy on a modern webapp, and wins hands down on time-to-first-data metric.

Re: Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser

#8

Post by platypii »

dmosites wrote: post #7

The iceberg reader sounds cool but how does it handle auth? Most iceberg tables are not publicly accessible.

It does support using S3 presigned requests, but it's admittedly a little awkward to ask a server for a presigned request before every fetch. But does still have the benefit that you can have a small and light server just handing out signed requests, and then the user and their browser does the heavy lifting. This can save a lot on scaling out server costs.

That being said, I wish there was a better auth story. Open to suggestions if anyone has ideas!

Re: Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser

#10

Post by abeppu »

Though these tools might be interesting, I wish they had called this something else. This isn't at all related to the concept of hyperparameters which people commonly refer to as hyperparams. And in their copy, the only reference to hyperparameters seems to be misusing the term.

> This stems from an industry-wide realization that model performance is ultimately bounded by data quality, not just model architecture or hyperparameters.

Generally we think of model architecture + weights (parameters) as making up the model itself, and hyperparam(s|eters) are the more relevant to how one arrives at those weights -- and for this reason are more relevant to the efficacy of training than the performance of the resultant model.

Post reply on HN