31–40 of 45 posts
Earlier quoted context omitted.
I'm also curious what's the use case of this over Ray. Tighter integration with PyTorch/tensors abstractions?
That. Also, it has RDMA. Last I checked, Ray did not support RDMA. There are probably other differences as well, but the lack of RDMA immediately splits the world into things you can do with ray and things you cannot do with ray
"Our Rust-based backend facilitates our performance, scale, and robustness — we amply use Rust’s fearless concurrency in Monarch’s implementation" Found a few typo's. The em dash makes me suspect an LLM was involved in proofreading
"Our Rust-based backend facilitates our performance, scale, and robustness — we amply use Rust’s fearless concurrency in Monarch’s implementation" Found a few typo's. The em dash makes me suspect an LLM was involved in proofreading
https://www.scottsmitelli.com/articles/em-dash-tool/
"Our Rust-based backend facilitates our performance, scale, and robustness — we amply use Rust’s fearless concurrency in Monarch’s implementation" Found a few typo's. The em dash makes me suspect an LLM was involved in proofreading
Apparently PyTorch oxidation has started. > Monarch is split into a Python-based frontend, and a backend implemented in Rust. Other than that, looks like a quite interesting project.
Multiple sources say that it is an experimental framework around PyTorch, not a replacement. People will still get to enjoy a circular graph using std::shared_ptr with memory leaks. It's a pity they don't do a complete rewrite with a functional language as the driver.
> ...Note that this does not support tensor engine, which is tied to CUDA and RDMA (via ibverbs).
I.e. yet another CUDA married approach: the issue is not ibverbs but the code shows they use GPUDirect RDMA, going from there this can only get worse - more CUDA dependencies. There would have been OpenUCX.
"Our Rust-based backend facilitates our performance, scale, and robustness — we amply use Rust’s fearless concurrency in Monarch’s implementation" Found a few typo's. The em dash makes me suspect an LLM was involved in proofreading
that it is surrounded by spaces makes this less likely
Earlier quoted context omitted.
that it is surrounded by spaces makes this less likely
Most style guides would call that an error, em dash should be used without surrounding spaces (while an en dash requires them). The only publication I know that has (recently?) eschewed that advice is WaPo. If the idea was to make it more visible, I believe the correct solution would have been for WaPo to use an en dash but render it longer in their typeface.
There are some infamous tech based on the "hiding" paradigm. PHP comes to mind. By hiding how the http request/response cycle actually works it fostered a generation of web developers who didn't know what a session cookie was, resulting in login systems that leaked like a sieve. Distributed computing is complicated. There are many parameters you need to tweak and many design decisions you need to take to make distributed model training run smoothly. I think explicit and transparent architectures are way better. Distributed model training shouldn't "feel" like running on a single device because it isn't.