Viewing profile — robertnishihara

To clarify, what I mean is that working with "exabytes" is atypical. Most use cases are at a slightly smaller scale :) Data processing workloads are quite common on Ray, especially…

Other folks have built data processing libraries on top of Ray: Modin and Daft come to mind. But I'm not aware of anything exactly like what you're referring to!

Multi-threaded libraries (e.g., numpy and PyTorch on CPUs come to mind) are well supported. In scenarios where many processes are each running heavily multi-threaded computations, …

I'm glad you find it exciting! Our intention from the start was for Ray to be general purpose. And the core Ray APIs are quite general (basically just scheduling a Python function …