Statistics that live in your SQL

I really appreciate the API for calling stats functions and retrieving results. This seamless integration was exactly what I was missing.

Regarding my concerns, the project currently gets you through about a first semester's worth of grad-level statistics or quantitative methods. It's sufficient for exploratory descriptive statistics, paired tests, and basic linear regressions. However, this also means it isn't close to the "production-level" statistics required for rigorous research work. At a minimum, it needs to support heteroskedastic-robust standard errors (Huber-White and clustered at a minimum; jackknife and bootstrapped as a non-parametric bonus), multilevel linear models, and generalized linear models (GLMs). I notice there is an open issue for GLM support, though it sounds like that will require a full rewrite of the inference backend. Including marginal estimates following a regression would also be highly useful, especially if GLMs are implemented.

Taking this library from an MVP to a production-ready replacement for SAS, R, or Stata will require significant effort. I am unsure about the market fit for a tool like this; organizations paying for SAS or Stata are unlikely to abandon them for an upstart project, and R has a deeply entrenched ecosystem that is impossible to replace. The situation feels similar to the dynamic between Octave and Matlab, or SageMath and Mathematica. It risks becoming a free alternative used primarily by those who cannot afford the paid products.

As others in the thread have pointed out, this extension includes functionality that is already handled well by other community extensions (like ggsql, stochastic, and read_stat). Because the ultimate goal seems to be providing a SAS-compatible frontend built on DuckDB, which is a huge and exciting undertaking. I wonder if the statistical backend might progress faster by focusing only on tests and regressions. Since it serves as the foundation for the other frontend services, zeroing in on the core stats and relying on the broader DuckDB ecosystem for the rest might make this massive scope a bit more manageable.

Agreed, it's not production-grade yet, and robust standard errors are the priority. HC and cluster-robust SEs are the biggest credibility gap, which is why I just added an (Eigen-based) linalg kernel for the next release: it's the groundwork so the regression fitter can ship HC0–HC3 and clustered SEs (and bootstrapped, via the existing bootstrap aggregate). Margins and GLMs (IRLS on the same kernel, so not really a backend rewrite) are the next layers.

About the scope, agreed, and already narrowing. I'll credit posit ggsql as the real grammar-of-graphics tool; mine stays a minimal built-in. The plan is to go deep on the statistical core and lean on the DuckDB ecosystem for the rest.

For the market fit I'd frame it a bit differently. Not trying to pry anyone off Stata/R for rigorous work, I know that those aren't going anywhere soon. The niche is stats where the data already lives: people in SQL/DuckDB who currently round-trip to R/Python for a regression. R can run in the browser now (webR), but not in the same engine as the data. The stats-duck runs inside duckdb-wasm, so there's no separate runtime and no marshalling. And DuckDB is already far faster than R for the ETL/wrangling around the stats.

The plan is to figure out from real usage. For now, I'll focus on the core.