PyTorch has been listing this install option for months, just click the "CUDA 11.8" button: https://pytorch.org/get-started/locally/
Yes, but 11.7 has been the "stable" release: https://github.com/pytorch/pytorch/blob/main/RELEASE.md#rele...
Use pytorch2+cu118 with ADA hardware for 50%+ speedup
11–20 of 38 posts
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#12wow if those benchmarks are true that is amazing to read.
Its true. I've been installing nightly builds of pytorch for months specifically to access this fix. Have been getting 40it/s outputting a 512x512 image on my 4090. Prior to the fix would get around 19it/s.
Why am I with a 3090 @ 3 it/s?
Am I doing something heavily wrong? All through WSL2
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#13Earlier quoted context omitted.
Its true. I've been installing nightly builds of pytorch for months specifically to access this fix. Have been getting 40it/s outputting a 512x512 image on my 4090. Prior to the fix would get around 19it/s.
Why am I with a 3090 @ 3 it/s? Am I doing something heavily wrong? All through WSL2
is/s depends on resolution and other factors like batch size. What are you getting for 512x image?
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#14Always cool to see :)
If you build from source, it should be even faster compared to release builds, if only because we keep on landing fixes and speedups regularly.
If anyone tries this and runs into bugs or issues, feel free to respond here and I can take a look.
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#15Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#16I accidentally stumbled upon this and did not expect such a speedup. It seems anything less than cu118 does not properly support the RTX4090 (or H100). Bumping to cuda12.2 with pytorch2.0.1+cu118 made my SDXL go 50% faster and ESRGAN 80% faster on the 4090.
You can also run PyTorch cu121 nightly builds, These also allow `torch.compile` to function properly with dynamic input, which should net another 30%+ boost to SD.
This comment brings a tear to my eye.
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#17I can confirm that it's true on RTX 4080 on Ubuntu 22.04 LTS.
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#18Eli5?
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#19Does it apply to Windows?
Re: Use pytorch2+cu118 with ADA hardware for 50%+ speedup
#20Eli5?
Using newer CUDA version with supported hardware and software boost performance