Does anyone know what kind of GPU support for Spark is mentioned in this announcement? Are they talking about existing XGBoost acceleration, or is it something general-purpose?
Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
151–160 of 347 posts
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#152Earlier quoted context omitted.
> AMD could release a 2x powerful GPGPU tomorrow for half the price and most current NVIDIA users wouldn't care because what good is that if you can't program it?. Correction: Nobody will be able to use the AMD hardware (outside of computer graphics) because everybody has been locked-in with CUDA on Nvidia. They can not even change even if they want to: it is pure madness to reprogram an entire GPGPU software stack e…
> Correction: Nobody will be able to use the AMD hardware (outside of computer graphics) because everybody has been locked-in with CUDA on Nvidia. NVIDA open-sourced their CUDA implementation to the LLVM project 5 years ago, which is why clang can compile CUDA today, and why Intel and PGI have clang forks compiling CUDA to multi-threaded and vectorized x86-64 using OpenMP. That you can't compile CUDA to AMD GPUs isn'…
> Do you work for AMD
I do not. And I use NVidia hardware regularly for GPGPU. But I hate fanboyism.
> NVIDA open-source their CUDA implementation to the LLVM project 5 years ago
Correction: Google developped an internal CUDA implementation for their own need based on LLVM that Nvidia barely supported it for their own need afterwards.
Nothing is "stable" nor "branded" in this work.... Consequently, 99% of public Open Source CUDA-using software still compile ONLY with the CUDA proprietary toolchain ONLY on NVidia hardware. And this is not going to change anything soon.
> one from PGI, that compile CUDA to multi-threaded x86-64 code using OpenMP.
The PGI compiler is proprietary and now property of NVidia. It was previously properitary and independant but mainly used for its GPGPU capability through OpenACC. OpenACC backend targets directly the nvidiaptx (proprietary) format. Nothing related with CUDA.
> Intel being the main vendor pushing for a parallel STL in the C++ standard
That's wrong again.
Most of the work done for the parallel STL and by the C++ committee originate from work from HPX and the STELLAR Group (http://stellar-group.org/libraries/hpx/).
They are pretty smart people and deserve at least respect and parent-ship for what they have done.
More information from Hermut Kaiser (Very Nice Guy btw) here (https://www.youtube.com/watch?v=6Z3_qaFYF84).
They have been the precursor of the idea of parallel "algorithms" in the STL and the concept of "Execution policy" you have in C++17 comes from them.
To the defense of Intel (and up to my knowledge) they have provided the first OSS implementation for compilers for it.
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#153No one is talking about it but the mention of a focus on "datacenter computing" (in the video) with CUDA is interesting. In retrospect a GPU is basically running a map/reduce type workflow so its not that crazy after all. Is this new or is CUDA already being used as a distributed language?
cuDF has supported Dask for distributed processing for a while now, maybe a year or two?
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#154Earlier quoted context omitted.
> Correction: Nobody will be able to use the AMD hardware (outside of computer graphics) because everybody has been locked-in with CUDA on Nvidia. NVIDA open-sourced their CUDA implementation to the LLVM project 5 years ago, which is why clang can compile CUDA today, and why Intel and PGI have clang forks compiling CUDA to multi-threaded and vectorized x86-64 using OpenMP. That you can't compile CUDA to AMD GPUs isn'…
> Do you work for AMD I do not. And I use NVidia hardware regularly for GPGPU. But I hate fanboyism. > NVIDA open-source their CUDA implementation to the LLVM project 5 years ago Correction: Google developped an internal CUDA implementation for their own need based on LLVM that Nvidia barely supported it for their own need afterwards. Nothing is "stable" nor "branded" in this work.... Consequently, 99% of public Open…
> That you can't compile CUDA to AMD GPUs isn't NVIDIA's fault, it's AMD, for deciding to pursue OpenCL first, then HSA, and now HIP.
Using a branded & under patent concurrent proprietary technology and copying its API for your own implementation is Maddness that will lead you for sure in front of a court.
It seems that even Google understood that the hard way (https://en.wikipedia.org/wiki/Google_v._Oracle_America)
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#155Earlier quoted context omitted.
Well, OP didn't say MKL works well on AMD. But you can at least run it on a non-Intel CPU. Compare CUDA.
The nvidia pgi compiler compiles CUDA to multi-core x86-64. There are other third-party compilers for CUDA->x86-64 (one LLVM-based one from Intel). There is a "library replacement" for CUDA from AMD called HIP, that you can use to map CUDA programs to ROCm. But... it doesn't work very well. NVIDIA also open-sourced CUDA support for Clang and LLVM. So anybody can extend clang to map CUDA to any hardware supported by L…
It sounds like people want nvidia to write drivers for AMD.
This criticism makes even less sense when any bystander could implement CUDA suppport on AMD by connecting open source software.
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#156Earlier quoted context omitted.
Well, OP didn't say MKL works well on AMD. But you can at least run it on a non-Intel CPU. Compare CUDA.
CUDA appears to have come out well before even OpenCL. I don't see why there would be expectation that nVidia would design their framework to work on a competitors product.
Cuda was also a response to ATI's own efforts at a proprietary effort which they eventually have up on.
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#157Important to remember the half-precision tensorcore misrepresentations where the 8x improvement over fp32 claimed on Imagenet with tensorcores (V100) was actually only 1.2-2x [1,2]. Furthermore, there are major precision issues with network architectures like variational autoencoders and many others.
We use V100s for Richardson-Lucy like deconvolutions for example, where we have near-exact photon counts up to 10,000 per pixel. fp32 is sufficient, tf32 is not.
V100 claimed 15 teraflops of FP32, A100 claims 19.5 teraflops. For most pytorch/tensorflow workflows out there, where FP32 dominates, this approximates closer to 30% improvement of last generation, which is reasonable and typical. Although FP64 does get a nice boost.
[1] https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v... [2] https://www.pugetsystems.com/labs/hpc/TensorFlow-Performance...
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#158Earlier quoted context omitted.
I don't like that people downvoted you for asking a question. If someone thinks the question is stupid or not doesn't mean that a downvote is warranted. (nor an upvote, answer the question and move on.) To answer though; it's just a coincidence, as you might already know Nvidia uses famous scientists (especially in the field of electricity) as the names of their microarchitectures. * Volta (Alessandro Volta, inventor…
Tesla is famous in the US for inventing polyphase AC and induction motors, but this is really one of these stories were a bunch of people invented the same thing very closely to each other due to a precipitating reaching of understanding. Note that Tesla's designs were IIRC two-phase which is largely inferior to three-phase. The push for three-phase and associated designs and inventions (three phase transformers on a…
> but this is really one of these stories were a bunch of people invented the same thing very closely to each other due to a precipitating reaching of understanding.
This is by far the dominant case of invention. Truly independent work is incredibly rare.
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#159I always thought that a real time scalable architecture would be beneficial. It's refreshing to see someone working on it, and exciting to see that it's nVidia. I always pictured a CPU with variable bit-width. Like a 256-bit ALU that could partition itself down into 16 or 32 bit ALU's as the workload allowed.
That's been around since MMX and AltiVec. It took a while for GPUs to adopt subword SIMD though.
Re: Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU
#160Earlier quoted context omitted.
Yeah. That's been my general problem with adopting NVidia for anything. They make good hardware, but there's a lot of lock-in, and not a lot of transparency. That introduces business risk. I'm not in a position where I need GPGPU, but if there wasn't that risk, and generally there were mature, open standards, I'd definitely use it. The major breakpoint would be when libraries like Numpy do it natively, and better yet…
Pretty much everyone these days uses a library for driving the GPU calculations. And they tend to either support multiple hardware targets directly (TensorFlow) or have API-compatible replacements (CuPy/NumPy). So the lock-in risk here is that you might have to run your stuff on CPU if future NVIDIA GPUs are too overpriced. I mean they are super expensive. But there's nothing that comes close to their cuBLAS library…
Yeah I think most people don't quite appreciate the difficulty and cost of optimizing for hardware and continually maintaining that through hardware cycles. In keeping things closed source Nvidia products have both the advantages of being easier to on-board due to simpler abstraction, and faster technical progress because there is less pushback from myriad parties when big inconvenient changes might need to happen at lower levels for hardware performance reasons kind of like if instead of x86 we instead settled on LLVM.