Who is providing the DMA engine in this case? Has the GPU access to PCIe device memory?
NVME-SSD performs as DMAC in this case. All GPU doing is mapping its own device memory on the PCI BAR area.
GpuScan and SSD-To-GPU Direct DMA
11–20 of 28 posts
Re: GpuScan and SSD-To-GPU Direct DMA
#12There is no explanation how it works. Does it work on top of existing APIs in user space? Or is there a custom kernel driver bypassing user space? I've done some high throughput streaming from HD/SSD to GPU before, and it's pretty easy to beat the naive solution but getting the most out of it would require kernel space code. I was doing random access streaming of textures using memory mapped files for input and copyi…
Its kernel module provides some special APIs. The userspase application (PostgreSQL) is enhanced to use them. From the point of user view, SQL still has been the interface to access the data.
Re: GpuScan and SSD-To-GPU Direct DMA
#13This is very awesome. If further developed + made into a feasible option for PostgreSQL, this has potential to do interesting things to TPC benchmarks. :)
Re: GpuScan and SSD-To-GPU Direct DMA
#14This would be incredibly useful for distributed machine learning - imagine a Tensorflow implementation that almost entirely bypasses CPU.
I was thinking the same thing, but is SSD to GPU faster than RAM to GPU? In many (not all) cases you buy a tonne of RAM and load your entire dataset into memory once and then iterate over it as necessary.
You also lose the flexibility of doing any sort of data modification or augmentation. One domain where your data usually doesn't fit in RAM is image recognition, but often you want to do things like apply random flips, crops and change hues before training to make the neural net less sensitive to those changes, which you can't really do with this.
Re: GpuScan and SSD-To-GPU Direct DMA
#15This would be incredibly useful for distributed machine learning - imagine a Tensorflow implementation that almost entirely bypasses CPU.
I was thinking the same thing, but is SSD to GPU faster than RAM to GPU? In many (not all) cases you buy a tonne of RAM and load your entire dataset into memory once and then iterate over it as necessary. You also lose the flexibility of doing any sort of data modification or augmentation. One domain where your data usually doesn't fit in RAM is image recognition, but often you want to do things like apply random fli…
SSD is probably not as fast as RAM, but it's much much cheaper, in the order of 10x per gigabyte. With SSD-GPU bridge you can have fast access to a multiple TiB training set, on a single machine.
Data pre-processing is indeed an issue, but hue adjustment/flipping/cropping could be implemented as Tensorflow operations, on the GPU. Similarly with input decompression - it would either have to be done on GPU, or the data would have to be stored uncompressed.
Re: GpuScan and SSD-To-GPU Direct DMA
#16I'm really hoping that Optane delivers on the hype, in which case our durable storage could be just 10x slower than RAM. At least, I imagine that it would be really helpful for speeding up even this approach.
Re: GpuScan and SSD-To-GPU Direct DMA
#17See also https://developer.nvidia.com/gpudirect and to some extent https://en.wikipedia.org/wiki/NVLink . NVLink is in the Power9 servers Google is using.
AFAIK Intel is stonewalling NVLink on their CPUs so they can (try to) sell Knight's landing. Quite a shame, although it might hurt then in the long run if they drive more institutes to buy ARM plus Tesla or Power plus Tesla clusters.
This is a perfect example of why we need healthy competition in the CPU market.
(Disclaimer: I don't have any way to verify whether the parent post is true, but I think the point stands regardless of whether this specific case is true or not.)
Re: GpuScan and SSD-To-GPU Direct DMA
#18Direct Direct Memory Access? That's pretty direct.
Re: GpuScan and SSD-To-GPU Direct DMA
#19Direct Direct Memory Access? That's pretty direct.
Redundancy makes it pretty indirect
Re: GpuScan and SSD-To-GPU Direct DMA
#20This would be incredibly useful for distributed machine learning - imagine a Tensorflow implementation that almost entirely bypasses CPU.
For most applications, getting training images onto the GPU isn't the bottleneck by far. Training the Inception model, for example, handles batches of 32 images (299x299x3) in 1.2 seconds. That's a pretty boring ~300KB * 32 ~= 10MB/sec of read bandwidth off the SSD for Imagenet. Even dealing with "real" images is probably only 10x that, which is trivial to get over the PCI bus.
The question would be whether we can turn the crank on the design of models to make it possible to do something really cool given access to very high-speed SSD storage.