11–17 of 17 posts
I see some potential wins on it, such as it being a Rust-based, Spark-compatible and better suited for single processor environments, but they are just not explained or developed enough.
Is this for distributed data processing like spark is? The documentation has nothing to indicate that it is.
> The mission of Sail is to unify stream processing, batch processing, and compute-intensive (AI) workloads. Currently, Sail features a drop-in replacement for Spark SQL and the Spark DataFrame API in single-process settings.
Bit off topic; we are looking for something like this but with a facility for untrusted users to run sandboxed code instead of trusted code. All that I found (but I am relatively new to this field) are hacky and, worse, slow solutions.
Bit off topic; we are looking for something like this but with a facility for untrusted users to run sandboxed code instead of trusted code. All that I found (but I am relatively new to this field) are hacky and, worse, slow solutions.
There is likes of comet and blaze that replace execution backend of spark with datafusion and then you have single process alternatives like sail trying to settle in "not so big data" category.
I am watching evolution of projects powered by datafusion and compatible with spark with keen eye. Early days but quite exciting.
Bit off topic; we are looking for something like this but with a facility for untrusted users to run sandboxed code instead of trusted code. All that I found (but I am relatively new to this field) are hacky and, worse, slow solutions.
What kind of sandboxing are you looking for, and what's the threat model? (My contact info is in my profile as well if you prefer)
nobody uses Spark for this outside of unit testing