this is nuts
We think so too, big things coming :)
Show HN: Attaching to a virtual GPU over TCP
41–50 of 117 posts
Re: Show HN: Attaching to a virtual GPU over TCP
#42Earlier quoted context omitted.
At the moment out tech is linux-only so it would not work with Blender. Down the line, we could see this being used for batched render jobs (i.e. to replace a render farm).
Blender can run on Linux…
Oh nice, I didn't know that! In that case it might work, you could try running `tnr run ./blender` (replace the ./blender with how you'd launch blender from the CLI) to see what happens. We haven't tested it so I can't make promises about performance or stability :)
Re: Show HN: Attaching to a virtual GPU over TCP
#43For anyone curious about how this actually works, it looks like a library is injected into your process to hook these functions [1] in order to forward them to the service.
Re: Show HN: Attaching to a virtual GPU over TCP
#44Ah this is quite interesting! I had a usecase where I needed a GPU-over-IP but only for transcoding videos. I had a not-so-powerful AMD GPU in my homelab server that somehow kept crashing the kernel any time I tried to encode videos with it and also an NVIDIA RTX 3080 in a gaming machine. So I wrote https://github.com/steelbrain/ffmpeg-over-ip and had the server running in the windows machine and the client in the me…
This is more or less what I was hoping for when I saw the submission title. Was disappointed to see that the submission wasn't actually a useful generic tool but instead a paid cloud service. Of course the real content is in the comments.
As an aside, are there any uses for GPU-over-network other than video encoding? The increased latency seems like it would prohibit anything machine learning related or graphics intensive.
Re: Show HN: Attaching to a virtual GPU over TCP
#45Reminds me of Plan9 :)
Re: Show HN: Attaching to a virtual GPU over TCP
#46I'm confused, if this operates at the CPU/GPU boundary doesn't it create a massive I/O bottleneck for any dataset that doesn't fit into VRAM? I'm probably misunderstanding how it works but if it intercepts GPU i/o then it must stream your entire dataset on every epoch to a remote machine, which sounds wasteful, probably I'm not getting this right.
Re: Show HN: Attaching to a virtual GPU over TCP
#47Earlier quoted context omitted.
We had a hell of a time dealing with the licensing issues and ultimately just gave up and give people whole GPUs. What are you doing to reset the GPU to clean state after a run? It's surprisingly complicated to do this securely (we're writing up a back-to-back sequence of audits we did with Atredis and Tetrel; should be publishing in a month or two).
We kill the process to reset the GPU. Since we only store GPU state that's the only clean up we need to do
Hm. Ok. Well, this is all very cool! Congrats on shipping.
Re: Show HN: Attaching to a virtual GPU over TCP
#48I'm confused, if this operates at the CPU/GPU boundary doesn't it create a massive I/O bottleneck for any dataset that doesn't fit into VRAM? I'm probably misunderstanding how it works but if it intercepts GPU i/o then it must stream your entire dataset on every epoch to a remote machine, which sounds wasteful, probably I'm not getting this right.
[deleted]
Re: Show HN: Attaching to a virtual GPU over TCP
#49I saw that in the tnr CLI, there are hints of an option to self-host a GPU. Is this going to be a released feature?
Re: Show HN: Attaching to a virtual GPU over TCP
#50I saw that in the tnr CLI, there are hints of an option to self-host a GPU. Is this going to be a released feature?
We don't support self-hosting yet but are considering adding it in the future. We're a small team working as hard as we can :)
Curious where you see this in the CLI, may be an oversight on our part. If you can join the Discord and point us to this bug we would really appreciate it!