I am working on developing a distributed operating system for managing machine learning resources (the entire memory hierarchy, networking and compute hierarchy). I worked in O/S development early in my career, now retired and ML is a hobby I've invested in over the past few years.
I'm curious about:
- how to "upcycle" end-of-life NVIDIA GPUs, like K80, M40, P100, etc.
- how to offload MoE expert calculations to MCUs with NPUs, possibly FPGAs
- how to manage a 10G LAN-attached network of older Xeon servers with the above GPUs and USB-attached MCUs as a machine learning "fabric"
- do ideas from IBM's ACP/TPF O/S (now Z/TPF) carry over?
- do ideas from IBM's Workload Manager (business priorities managed resources) carry over?
- can I bypass end-of-life CUDA driver support for older NVIDIA hardware by using Rust/Vulcan?
- can I leverage the architecture of Freetoken (MoE caching/routing, Engram, KV-cache compression) and other approaches like Baby Dragon Hatchlings, Hierarchical Reasoning Models, Tiny Recursive Models, Recurisve Language Models, Multi-token predicion, etc?
- is anyone else intersted in actually building something like this?
I've explored many of these parts individually, now combining them...