Use pytorch2+cu118 with ADA hardware for 50%+ speedup
1–10 of 38 posts
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#2I accidentally stumbled upon this and did not expect such a speedup. It seems anything less than cu118 does not properly support the RTX4090 (or H100).
Bumping to cuda12.2 with pytorch2.0.1+cu118 made my SDXL go 50% faster and ESRGAN 80% faster on the 4090.
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#3[dead]
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#4wow if those benchmarks are true that is amazing to read.
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#5Can the same speedup be obtained on a 3090?
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#6I accidentally stumbled upon this and did not expect such a speedup. It seems anything less than cu118 does not properly support the RTX4090 (or H100). Bumping to cuda12.2 with pytorch2.0.1+cu118 made my SDXL go 50% faster and ESRGAN 80% faster on the 4090.
Good find!
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#7I accidentally stumbled upon this and did not expect such a speedup. It seems anything less than cu118 does not properly support the RTX4090 (or H100). Bumping to cuda12.2 with pytorch2.0.1+cu118 made my SDXL go 50% faster and ESRGAN 80% faster on the 4090.
You can also run PyTorch cu121 nightly builds,
These also allow `torch.compile` to function properly with dynamic input, which should net another 30%+ boost to SD.
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#8PyTorch has been listing this install option for months, just click the "CUDA 11.8" button:
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#9wow if those benchmarks are true that is amazing to read.
Its true. I've been installing nightly builds of pytorch for months specifically to access this fix. Have been getting 40it/s outputting a 512x512 image on my 4090. Prior to the fix would get around 19it/s.
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#10PyTorch has been listing this install option for months, just click the "CUDA 11.8" button: https://pytorch.org/get-started/locally/
[deleted]