Viewing profile — yeshengm

Since you need fast lookup, traditional transactional row store like RocksDB seems a better fit.

Are these execution engines internally using Arrow columnar format or are they just exposing Arrow as a client wire format? AFAIK Spark and Presto does not use Arrow as execution c…

There are a lot of AI compilers out there: MLIR/TVM/XLA/Halide/Hgraph/TensorRT etc... Do we really need so many AI compilers given that only PyTorch/Tensorflow/MXNet are mainsteam …

Yeah. Likely a small amount of analytics jobs taking up most memory/CPU footprint, while critical prod OLTP jobs take very little resource comparably.

Nowadays most Spark users use either PySpark or Spark SQL. Spark is designed in some sense to mimic Scala collection library. Rust is nice but its interop with other open source bi…

I'd say start with some eager functional language such as OCaml without the "object" part. Haskell code is hard to debug cuz you can't rely on good old print. Monad itself is never…