1–10 of 117 posts
Show HN: Attaching to a virtual GPU over TCP
thundercompute.com
Hmm... well I just watched you run nvidia-smi in a Mac terminal, which is a platform it's explicitly not supported on. My instant assumption is that your tool copies my code into a private server instance and communicates back and forth to run the commands.
Does this platform expose eGPU capabilities if my host machine supports it? Can I run raster workloads or network it with my own CUDA hardware? The actual way your tool and service connects isn't very clear to me and I assume other developers will be confused too.
this is awesome. can it do 3d rendering (vulkan/opengl)
> Access serverless GPUs through a simple CLI to run your existing code on the cloud while being billed precisely for usage Hmm... well I just watched you run nvidia-smi in a Mac terminal, which is a platform it's explicitly not supported on. My instant assumption is that your tool copies my code into a private server instance and communicates back and forth to run the commands. Does this platform expose eGPU capabil…
Going into more details for how this works, we intercept communication between the CPU and the GPU so only GPU code and commands are sent across the network to a GPU that we are hosting. This way we are able to virtualize a remote GPU and make your computer think it's directly attached to that GPU.
We are not copying your CPU code and running it on our machines. The CPU code runs entirely on your instance (meaning no files need to be copied over or packages installed on the GPU machine). One of the benefits of this approach is that you can easily scale to a more / less powerful GPU without needing to setup a new server.
> Access serverless GPUs through a simple CLI to run your existing code on the cloud while being billed precisely for usage Hmm... well I just watched you run nvidia-smi in a Mac terminal, which is a platform it's explicitly not supported on. My instant assumption is that your tool copies my code into a private server instance and communicates back and forth to run the commands. Does this platform expose eGPU capabil…
Great questions! To clarify the demo, we were ssh'd into a linux machine with no GPU. Going into more details for how this works, we intercept communication between the CPU and the GPU so only GPU code and commands are sent across the network to a GPU that we are hosting. This way we are able to virtualize a remote GPU and make your computer think it's directly attached to that GPU. We are not copying your CPU code a…
will that cause system instability, say, if the network suddenly dropped?
is this a remote nvapi? this is awesome. can it do 3d rendering (vulkan/opengl)
is this a remote nvapi? this is awesome. can it do 3d rendering (vulkan/opengl)
> is this a remote nvapi
Essentially yes! Just to be clear, this covers the entire GPU not just the NVAPI (i.e. all of cuda). This functions like you have the physical card directly plugged into the machine.
Right now we don't support vulkan or opengl since we're mostly focusing on AI workloads, however we plan to support these in the future (especially if there is interest!)
Earlier quoted context omitted.
Great questions! To clarify the demo, we were ssh'd into a linux machine with no GPU. Going into more details for how this works, we intercept communication between the CPU and the GPU so only GPU code and commands are sent across the network to a GPU that we are hosting. This way we are able to virtualize a remote GPU and make your computer think it's directly attached to that GPU. We are not copying your CPU code a…
does this mean you have a customized/dummy kernel gpu driver? will that cause system instability, say, if the network suddenly dropped?
Given that, if the network suddenly dropped then only the process using the GPU would fail.
is this a remote nvapi? this is awesome. can it do 3d rendering (vulkan/opengl)
Thank you! > is this a remote nvapi Essentially yes! Just to be clear, this covers the entire GPU not just the NVAPI (i.e. all of cuda). This functions like you have the physical card directly plugged into the machine. Right now we don't support vulkan or opengl since we're mostly focusing on AI workloads, however we plan to support these in the future (especially if there is interest!)
I bet you saw this https://github.com/mikex86/LibreCuda
they implemented the cuda driver by calling into rmapi.
My understanding is if there is a remote rmapi, other user mode drivers should work out of the box?