61–70 of 125 posts
Interesting that DLSS 3 is mentioned as an advantage?
Thanks for sharing. Have you prodded the model with various inputs and written an article that show various output examples? I'd love to get an idea of what sort of "end product" 4x4090s is capable of producing.
Earlier quoted context omitted.
Here is some additional journey apart from the rig. https://sabareesh.com/posts/llm-intro/
How long does training a 1B or 500M model take approximately on the 4-GPU setup? Or does that dramatically depend on the training data? I didn’t see that info on your pages.
You can get 4060 ti 16GB cards for ~$450 or 4070 ti 16gb for ~850 instead of the $2.5k for a 4090. I wonder how well 4 of those cards would perform. The 4060 TDP is 165w instead of 450w for the 4090. The 4070 looks like the best tradeoff though for cost/power/etc though. You could probably set up an 8 card 4070 ti 16gb system for less than the 4 card 4090 system
Let's talk riser cables. I keep encountering issues with riser connectors claiming to support PCIe 4.0, which seem to have sub-par performance. They work fine with the GPUs and NICs I tested them with, but attaching a nvme drive causes all kinds of issues and prevents the machine from booting. I guess nvme isn't as tolerant of elevated bit-error-rates. That just doesn't inspire a lot of confidence in those risers, so…
You can get 4060 ti 16GB cards for ~$450 or 4070 ti 16gb for ~850 instead of the $2.5k for a 4090. I wonder how well 4 of those cards would perform. The 4060 TDP is 165w instead of 450w for the 4090. The 4070 looks like the best tradeoff though for cost/power/etc though. You could probably set up an 8 card 4070 ti 16gb system for less than the 4 card 4090 system
I was eyeing 4060 before going with 4090. But it boils down to cuda cores and memory bandwidth
Earlier quoted context omitted.
The GPU rental market is fairly reasonable. There's lots of companies doing it. (I work at one of them). 4x 4090 can be fetched for around $0.40/hour on some platforms ... about $1.20 on others depending on how available you want it. Regardless, all in, you can do an average 10-or-so-day train for If you want on-prem, wait a few months. The supply of 5000 series (probably announced at CES in a few days) should push m…
How soon could I break even on renting my GPUs out?
It's probably somewhere between 12months-never depending on how the market shakes out. Maybe 2 years is a good idea ... really, if power is cheap/free and the machine is on and idle then it's free money - that's the way to look at it.
Earlier quoted context omitted.
How soon could I break even on renting my GPUs out?
is your electricity free? Some of these cards probably cost about $0.10/hr to run ... depending on your card/electricity rate etc. It's probably somewhere between 12months-never depending on how the market shakes out. Maybe 2 years is a good idea ... really, if power is cheap/free and the machine is on and idle then it's free money - that's the way to look at it.