Viewing profile — platypii

[flagged]

Sylvain Lesage’s cool interactive explainer on visualizing extreme row counts—think billions—inside the browser. His technical deep dive explains how the open-source library HighTa…

We're willing to spend money, but I've had the "datadog billing problem" before where it starts reasonable and then grows to a non-trivial percent of saas budget, and then theres a…

LLM logs are crushing my application logging system. We recently launched AI features on our app and went from ~100mb/month of normal website logs to 3gb/month of llm conversation …

I made a small (~9 KB), open source SQL engine in JavaScript built for interactive data exploration. Squirreling is unique in that it’s built entirely with modern async JavaScript …

I asked this on the Apache mailing list but haven’t found a good solution yet. Wondering if anyone has some ideas for how to engineer this? Here’s my problem: I have gigabytes of L…

I’ve been doing data science for years, and am very familiar with jupyter notebooks and more recently been using duckdb a lot. But now I have this huge pile of output tokens from m…

Makes sense. I'm not currently in snowflake because I'm mostly working with local parquet files. Would prefer not to have to pay for snowflake just to explore my data. I'm interest…

Tools like DuckDB Wasm and data engineering platforms like Iceberg leverage Parquet’s built-in indexing to very efficiently query files over the network. But as I’ve been building …

I started Hyperparam one year ago because I knew that the world of data was changing, and existing tools like Python and Jupyter Notebooks were not built for the scale of LLM data.…

There’s an important problem with AI that nobody’s talking about. AI’s entire lifecycle is tons of data in for training, and an even larger amount of text data out. Traditional too…

This is a Q&A I did on what I learned from a year of open source data transformation. Most of all, it reinforced my belief that browser-native tools aren’t “toys” that don’t work f…

As with anything, there are engineering tradeoffs. What I've found is that moving data processing toward the browser has been for one, a refreshing developer experience because I d…

How far can we push the browser as a data engine — not just for visualizations, but for curating and querying large datasets? Do we need traditional backend architectures? I wanted…

Why not? We are trying to evaluate AI's capabilities. It's OBVIOUS that we should compare it to our only prior example of intelligence -- humans. Saying we shouldn't compare or ant…

This is the story of how I spent a year making the world's fastest Parquet loader in JavaScript. The goal: - Make a faster, more interactive viewer for AI datasets (which are mostl…

I don’t have benchmarks specifically against duckdb. I’m sure native C++ will run faster than JavaScript. But whats important is that with Hyperparam you can do it in the browser, …

Funny you say that, because I built these tools because I wanted to build something very much like what you're describing! I was trying to look at, filter, and transform large AI d…

Yea except with parquet you don't need to load the entire file, the parquet metadata let's you do http range requests for just the data you need. For example this parquet is the en…

That's fair criticism... to be honest when I started the project it was more focused on hyperparameters, and it evolved into this javascript-for-ai mission. But now I just kind of …

It does support using S3 presigned requests, but it's admittedly a little awkward to ask a server for a presigned request before every fetch. But does still have the benefit that y…