I’m more interested in using tools like hashcat, any benchmark on these? As the docs link returns error.
Show HN: Attaching to a virtual GPU over TCP
91–100 of 117 posts
Re: Show HN: Attaching to a virtual GPU over TCP
#92Ah this is quite interesting! I had a usecase where I needed a GPU-over-IP but only for transcoding videos. I had a not-so-powerful AMD GPU in my homelab server that somehow kept crashing the kernel any time I tried to encode videos with it and also an NVIDIA RTX 3080 in a gaming machine. So I wrote https://github.com/steelbrain/ffmpeg-over-ip and had the server running in the windows machine and the client in the me…
This is more or less what I was hoping for when I saw the submission title. Was disappointed to see that the submission wasn't actually a useful generic tool but instead a paid cloud service. Of course the real content is in the comments. As an aside, are there any uses for GPU-over-network other than video encoding? The increased latency seems like it would prohibit anything machine learning related or graphics inte…
I am increasingly growing tired of these "cloud" services, paid or not. :/
Re: Show HN: Attaching to a virtual GPU over TCP
#93Ah this is quite interesting! I had a usecase where I needed a GPU-over-IP but only for transcoding videos. I had a not-so-powerful AMD GPU in my homelab server that somehow kept crashing the kernel any time I tried to encode videos with it and also an NVIDIA RTX 3080 in a gaming machine. So I wrote https://github.com/steelbrain/ffmpeg-over-ip and had the server running in the windows machine and the client in the me…
Interesting. Do you know if your tool supports conversions resulting in multiple files, such as HLS and its myriad of timeslice files?
Since it’s sharing the underlying file system and just running ffmpeg remotely, it should support any variation of outputs
Re: Show HN: Attaching to a virtual GPU over TCP
#94Earlier quoted context omitted.
That understanding of the system is correct. To make it practical we've implemented a bunch of optimizations to minimize I/O cost. You can see how it performs on inference with BERT here: https://youtu.be/qsOBFQZtsFM?t=69 . The overheads are larger for training compared to inference, and we are implementing more optimizations to approach native performance.
> to approach native performance. The same way one "approaches the sun" when they take the stairs?
I guess there's non-negligible optimization potential, e.g. by doing hash-based caching. If the same data gets uploaded twice they can have the blob already sitting somewhere closer to the machine.
Re: Show HN: Attaching to a virtual GPU over TCP
#95[deleted]
Re: Show HN: Attaching to a virtual GPU over TCP
#96For anyone curious about how this actually works, it looks like a library is injected into your process to hook these functions [1] in order to forward them to the service. [1] https://pastebin.com/raw/kCYmXr5A
Ah, I assumed/hoped they had some magic that would manage to forward a whole PCIe device.
Re: Show HN: Attaching to a virtual GPU over TCP
#97Earlier quoted context omitted.
This is more or less what I was hoping for when I saw the submission title. Was disappointed to see that the submission wasn't actually a useful generic tool but instead a paid cloud service. Of course the real content is in the comments. As an aside, are there any uses for GPU-over-network other than video encoding? The increased latency seems like it would prohibit anything machine learning related or graphics inte…
I am increasingly growing tired of these "cloud" services, paid or not. :/
Well, feel free to spend your own time on writing such a tool and releasing it as Open Source. That would be a really cool project! Until then, don't complain that others aren't willing to donate a significant amount of their work to the public.
Re: Show HN: Attaching to a virtual GPU over TCP
#98Earlier quoted context omitted.
Okay, but your GPUs are in ECS. Don't I just want this feature from Amazon, not you, and natively via Nitro? Or even Google has TPU attachments. > 1. If you're actively developing and need a GPU [for fractional amounts of time]... Why would I need a GPU for a short amount of time during development? For testing? I don't get it - what would testing an H100 over a TCP connection tell me? It's like, yeah, I can do that,…
I develop GPU accelerated web apps in an EC2 instance with a remote VSCode session. A lot of the time I’m just doing web dev and don’t need a GPU. I can save thousands per month by switching to this.
Sounds like you can save thousands by just buying a simple GPU card.
Re: Show HN: Attaching to a virtual GPU over TCP
#99Even a directly attached eGPU via thunderbold 4 was after some time too slow for machine learning aka training. As i work now fully remote i just have a beefy midi tower. Some context about eGPU [1].
But hey i'm happy to be proofed wrong ;)
Re: Show HN: Attaching to a virtual GPU over TCP
#100Earlier quoted context omitted.
I develop GPU accelerated web apps in an EC2 instance with a remote VSCode session. A lot of the time I’m just doing web dev and don’t need a GPU. I can save thousands per month by switching to this.
Sounds like you can save thousands by just buying a simple GPU card.
Well, for the time being I'm really just burning AWS credits. But you're right! I do however like that my dev machine is the exact same instance type in the same AWS region as my production instances. If I built an equivalent machine it would have different performance characteristics. Often times the AWS VMs have weird behavior that I would otherwise be caught off guard with when deploying to the cloud for the first time.