Live data from Hacker News

Show HN: Parqeye – A CLI tool to visualize and inspect Parquet files

github.com

11–20 of 37 posts

Re: Show HN: Parqeye – A CLI tool to visualize and inspect Parquet files

#11

It’s crazy how long we’ve gone without a tool like this. This is huge. Thank you for finally building this!

It is really incredible how poor the parquet tooling has been for years. The cornerstone of data engineering, yet just inspecting a file is needlessly clunky.

Re: Show HN: Parqeye – A CLI tool to visualize and inspect Parquet files

#15
Nice work—this hits a real pain point with Parquet. My main use case is debugging partitioned datasets on S3 with schema drift and skew, where I care about: which files/partitions have schema mismatches, weird row-group stats (all-null, out-of-range, huge skew), and doing that via metadata only.

Right now parqeye looks mainly single-file focused. Do you have plans for a “dataset mode” that takes a dir/S3 prefix and surfaces per-file/row-group summaries (row counts, min/max, null %, schema diffs vs a reference file) using just Parquet stats so it scales to tens of GB? Or do you see parqeye intentionally staying a single-file inspector?

Re: Show HN: Parqeye – A CLI tool to visualize and inspect Parquet files

#18

thank you so much! this was an annoyance of mine for so long. edit: any chance you make a brew package? if you'd like I'd be happy to PR it in.

yep! it’s available as a homebrew tap — you can install it with: `brew install kaushiksrini/parqeye/parqeye`

awesome! i was just looking at a bucket full of parquet files from last year trying to recall some things about them.

i tried to install with brew, but it told me my cli tools were "too out of date". Never seen that before! and also just upgraded.

Will try again tomorrow

Re: Show HN: Parqeye – A CLI tool to visualize and inspect Parquet files

#20
Yours looks much better for your use case, but fwiw you can do it in a single command with duckdb too (but not interactive etc.):

    duckdb -c "from 'foo.parquet'"

but maybe still useful for other formats or multi-file or remote situations
Post reply on HN