31–40 of 65 posts
This is great. The BlazingDB guys are awesome and now that the project is open source this is another good reason for my teams to experiment with different workloads and compare it against a SparkSQL approach
This is great. The BlazingDB guys are awesome and now that the project is open source this is another good reason for my teams to experiment with different workloads and compare it against a SparkSQL approach
+1, this is very cool, but would love for the BlazingDB team to show benchmarks here
Check it out, it's fast.
Looks like an good way to do analytics on the GPU. The Python API is clean and simple. The premise is that GPUs will accelerate columnar data analytics. And, with "Dask" [1], you can run those worldloads on a cluster. I wonder if careful indexing on initial write would outperform this system. This system looks like it's best when you have totally raw, unindexed data. Perhaps a future thing to do is to generate a side…
What kind of benefits does CUDA bring to databases? I've never heard of running a database on a GPU before. Couldn't find anything on their homepage other than comparison with a few other db options
This seems pretty cool, and I'll probably play with this at some point, but sadly literally all of my GPUs are AMD or Intel at this point. I'm sure you had a good reason, so I'm genuinely curious to why CUDA was chosen instead of something like OpenCL? (I'll add my typical disclaimer that I'm not saying this as some passive-aggressive way to criticize; I'm genuinely curious to the reasoning behind the choice.)
CUDA has two APIs: 1. The runtime api (libcudart.so) 2. The driver api (libcuda.so). The driver api is very close to the opencl api and is very low level. Most people use the CUDA runtime api which is vastly more convenient. The main difficulty with OpenCl and the driver api is that you have to manually load GPU code onto the device which then returns a handle. You generally have to load the code onto every device wh…
What kind of benefits does CUDA bring to databases? I've never heard of running a database on a GPU before. Couldn't find anything on their homepage other than comparison with a few other db options
This is a Distributed SQL engine not a database. We store no data. You store your data in HDFS, S3, posix, NFS etc. We allow you to query directly from these filesystems of the file formats you have already. You can look here to see the file formats cudf supports. https://github.com/rapidsai/cudf/tree/branch-0.9/cpp/src/io You can try it out yourself here https://colab.research.google.com/drive/1r7S15Ie33yRw8cmET7_..…
Is it distributed? How do I set it up in a distributed mode? Does it support nested parquet (something that even spark itself struggles to support inside SQL).
Earlier quoted context omitted.
This is a Distributed SQL engine not a database. We store no data. You store your data in HDFS, S3, posix, NFS etc. We allow you to query directly from these filesystems of the file formats you have already. You can look here to see the file formats cudf supports. https://github.com/rapidsai/cudf/tree/branch-0.9/cpp/src/io You can try it out yourself here https://colab.research.google.com/drive/1r7S15Ie33yRw8cmET7_..…
I've read the website, but I could't find a hint that the engine is distributed. Even the spark benchmarks compare a single instance with multiple nodes. Is it distributed? How do I set it up in a distributed mode? Does it support nested parquet (something that even spark itself struggles to support inside SQL).
Right now we use k8s on Google K8s Engine(GKE) to deploy in distributed mode.
We don't supported nested at present, there are Rapids teams looking into this.
Earlier quoted context omitted.
CUDA has two APIs: 1. The runtime api (libcudart.so) 2. The driver api (libcuda.so). The driver api is very close to the opencl api and is very low level. Most people use the CUDA runtime api which is vastly more convenient. The main difficulty with OpenCl and the driver api is that you have to manually load GPU code onto the device which then returns a handle. You generally have to load the code onto every device wh…
The open conccurent to the runtime api is SYCL.
This seems pretty cool, and I'll probably play with this at some point, but sadly literally all of my GPUs are AMD or Intel at this point. I'm sure you had a good reason, so I'm genuinely curious to why CUDA was chosen instead of something like OpenCL? (I'll add my typical disclaimer that I'm not saying this as some passive-aggressive way to criticize; I'm genuinely curious to the reasoning behind the choice.)
CUDA has two APIs: 1. The runtime api (libcudart.so) 2. The driver api (libcuda.so). The driver api is very close to the opencl api and is very low level. Most people use the CUDA runtime api which is vastly more convenient. The main difficulty with OpenCl and the driver api is that you have to manually load GPU code onto the device which then returns a handle. You generally have to load the code onto every device wh…
They are only realistically comparable from OpenCL 2.0 onwards. But no NVIDIA card supports anything beyond 1.2, and with that decision they basically killed OpenCL.
This seems pretty cool, and I'll probably play with this at some point, but sadly literally all of my GPUs are AMD or Intel at this point. I'm sure you had a good reason, so I'm genuinely curious to why CUDA was chosen instead of something like OpenCL? (I'll add my typical disclaimer that I'm not saying this as some passive-aggressive way to criticize; I'm genuinely curious to the reasoning behind the choice.)
Thats a great question. The answer is two-fold. Early on when we first started playing around with General Processing on GPU's we had Nvidia cards to begin with and I started looking at the apis that were available to me. The CUDA ones were easier for me to get started, had tons of learning content that Nvidia provided, and were more performant on the cards that I had at the time compared to other options. So we buil…
Were some benchmarks done perhaps or could you provide some more low-level reasons as to why CUDA was more performant? I'm not experienced with CUDA, just generally interested.
I also have to say that I am a bit skeptical of Nvidia as I have never received any proper support for Linux development on Nvidia GPUs for drivers and generally tracking bugs on their cards. It was so frustrating that I just switched to AMD GPUs that "just worked". How is this different for these kinds of use cases? Does Nvidia only care about their potential enterprise customers but they don't care about general usage of their GPUs on Linux? It seems to rub me the wrong way and I don't understand.