21–30 of 37 posts
Earlier quoted context omitted.
The benchmarks where the 1080 doesn't even compete - FP16/Tensor Cores.
Many state-of-art models won't train well on FP16. But for inferencing it's extraordinarily good. 2x1080Ti is the sweet spot for FP32 training on "budget" at the moment.
Earlier quoted context omitted.
Many state-of-art models won't train well on FP16. But for inferencing it's extraordinarily good. 2x1080Ti is the sweet spot for FP32 training on "budget" at the moment.
Got any sources? Was thinking about buying one just for the tensor cores, but if this is the case I probably won't.
"When I first looked at fp16 Inception3 was the largest model I could train. Inception4 blew up until I went back to fp32. Mixed precision needs extra care, scaling of gradients and such. Still I think it is a good thing. What I really want to test is model size reduction for inference with TensorRT targeted to tensorcores. I think that is probably the best use case. Non-linear optimization is just too susceptible to precision loss."
There was also some NVidia video presentation recommending mixed FP32/FP16 training instead of pure FP16.
Are there benefits to using FP32 vs FP16? I’ve been dabbling with deep learning but not really sure how much affect higher precision is having. Though more precision is better I suppose.
With FP16 one can theoretically get 2x speed and 2x larger models with the same VRAM capacity. For inferencing with INT8/INT4 it can be even way better (good for embedded stuff). The downside is that sometimes more complex/deep models don't converge (or converge less often than FP32). Sometimes there are framework issues with some advanced FP16 stuff.
Does anyone know why they are using Xeon processors instead of the AMD Threadripper? Is it the support for ECC memory? If so, why is it that so important? Example: https://www.pugetsystems.com/nav/peak/tower_single/customize...
Because it's a single-root pci complex. See here: https://www.servethehome.com/how-intel-skylake-sp-changes-im...
[1] https://www.intel.com/content/www/us/en/processors/xeon/xeon...
Earlier quoted context omitted.
Got any sources? Was thinking about buying one just for the tensor cores, but if this is the case I probably won't.
You can even see it in author's comments in the original article: "When I first looked at fp16 Inception3 was the largest model I could train. Inception4 blew up until I went back to fp32. Mixed precision needs extra care, scaling of gradients and such. Still I think it is a good thing. What I really want to test is model size reduction for inference with TensorRT targeted to tensorcores. I think that is probably the…
Does anyone know why they are using Xeon processors instead of the AMD Threadripper? Is it the support for ECC memory? If so, why is it that so important? Example: https://www.pugetsystems.com/nav/peak/tower_single/customize...
These numbers match up with the performance that we’ve measured in our own tests that were posted last week. The Titan V is simply too expensive for Deep Learning. The 2080 TI is, by far and away, the best GPU from a price/performance perspective.
As mentioned in the article, only possible reason that you might want a Titan V is if you care about FP64 performance: i.e., nobody training neural networks.
Does anyone know why they are using Xeon processors instead of the AMD Threadripper? Is it the support for ECC memory? If so, why is it that so important? Example: https://www.pugetsystems.com/nav/peak/tower_single/customize...
The Xeon-W 2175 has avx-512. Threadrippers cannot compete in numbering work relative to price point on well optimized code. sgemm on 5000x5000 matrices takes about 600ms on a Threadrippers 1950x, but only around 150ms on the comparatively priced i9 7900x. Vector libraries for special functions, eg Intel VML or SLEEF also provide a similar performance advantage there. If you're mostly crunching numbers, and either com…
I don't follow the logic. It sounds like you're saying that if you care about that specific type of highly vectorized computation being fast what you really want is a GPU rather than any particular CPU. So how should that have a major influence on which CPU you choose? Particularly when the CPU which is slower at that is faster at many other things that aren't suitable for a GPU.
Does anyone know why they are using Xeon processors instead of the AMD Threadripper? Is it the support for ECC memory? If so, why is it that so important? Example: https://www.pugetsystems.com/nav/peak/tower_single/customize...
The Xeon they are using is 14 cores in a single NUMA node so maybe that's it since a 16 core Threadripper is 2 separate nodes in one die. I'm pretty sure Threadripper supports ECC: "With the most memory channels you can get on desktop, the Ryzen™ Threadripper™ processor can support Workstation Standard DDR4 ECC (Error Correcting Mode) Memory to keep you tight, tuned and perfectly in sync." from https://www.amd.com/en…