What UI do you use on top of data engineering tools to look at data?

#1

Post by platypii » Wed, Dec 10, 2025, 6:10 PM UTC

Tools like DuckDB Wasm and data engineering platforms like Iceberg leverage Parquet’s built-in indexing to very efficiently query files over the network. But as I’ve been building data tools myself, the stack gets complicated fast, especially once you try to visualize or explore the data instead of just querying it. I’m intrigued by some of the modern tricks people are using to do more data engineering client-side.

With OPFS + Parquet + Wasm, the browser already has everything it needs to handle multi-GB LLM datasets client-side.

Is the world of data UIs evolving? Are there new data tools and best practices beyond notebooks and DuckDB?

Re: What UI do you use on top of data engineering tools to look at data?

#2

Post by rawgabbit » Wed, Dec 10, 2025, 6:21 PM UTC

Having tried a variety of tools, I usually return to Snowflake. Within Snowflake, I don't use notebooks; instead, I prefer to create database Views to contain my code. I may have Views that call other Views etc. I like Snowflake because for every query in its UI, it automatically profiles the results it returns. e.g., for each column it gives a frequency distribution histogram of the values within the column. In short, I like the ability to store my code in something I can easily backup and re-use without adding a lot of other tools.

Re: What UI do you use on top of data engineering tools to look at data?

#3

Post by platypii » Wed, Dec 10, 2025, 6:35 PM UTC

rawgabbit wrote: post #2

Having tried a variety of tools, I usually return to Snowflake. Within Snowflake, I don't use notebooks; instead, I prefer to create database Views to contain my code. I may have Views that call other Views etc. I like Snowflake because for every query in its UI, it automatically profiles the results it returns. e.g., for each column it gives a frequency distribution histogram of the values within the column. In shor…

Makes sense. I'm not currently in snowflake because I'm mostly working with local parquet files. Would prefer not to have to pay for snowflake just to explore my data. I'm interested in better data UIs though so I might need to check it out.

Re: What UI do you use on top of data engineering tools to look at data?

#4

Post by rawgabbit » Wed, Dec 10, 2025, 11:11 PM UTC

platypii wrote: post #3

rawgabbit wrote: post #2

Having tried a variety of tools, I usually return to Snowflake. Within Snowflake, I don't use notebooks; instead, I prefer to create database Views to contain my code. I may have Views that call other Views etc. I like Snowflake because for every query in its UI, it automatically profiles the results it returns. e.g., for each column it gives a frequency distribution histogram of the values within the column. In shor…

Makes sense. I'm not currently in snowflake because I'm mostly working with local parquet files. Would prefer not to have to pay for snowflake just to explore my data. I'm interested in better data UIs though so I might need to check it out.

To keep costs down. I use standard edition with x-small data warehouse. The biggest cost driver is CPU or how long queries run.