Show HN: Hardware-software co-design for MoE models to bypass NCCL bottlenecks
1–2 of 2 posts
Re: Show HN: Hardware-software co-design for MoE models to bypass NCCL bottlenecks
#2Imagine virtual memory, but instead of using your local storage, you’re pooling a whole bunch of network servers into a single address space