Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BlazingSQL is Now Open Source

blog.blazingdb.com

Post reply on HN
Print view View original

51–60 of 65 posts

Re: BlazingSQL is Now Open Source

#51

Post by rrss »

tombert wrote: post #20
felipe_aramburu wrote: post #17

Earlier quoted context omitted.

Thats a great question. The answer is two-fold. Early on when we first started playing around with General Processing on GPU's we had Nvidia cards to begin with and I started looking at the apis that were available to me. The CUDA ones were easier for me to get started, had tons of learning content that Nvidia provided, and were more performant on the cards that I had at the time compared to other options. So we buil…

Thanks for answering my question so quickly! That seems like a pretty good reason...I have been looking to learn some GPU programming to optimize some matrix math that I've been doing for a pet project, and while my first instinct was telling me OpenCL since it's portable, if people who actually know what they're talking about are saying that CUDA is simpler to start with, it might be worth it to me to pick up a chea…

> OpenCL since it's portable

Even if you choose OpenCL, the tools (profiler, debugger, etc) are usually platform specific. In addition, my experience with opencl across platforms was that each of the vendors' compilers had distinct issues and that performance was not portable.

I get the appeal for an open API, but opencl never grew a development ecosystem or any libraries. IMO it is dying and isn't worth the effort. AMD is implementing CUDA with hip - maybe roll with that.

Re: BlazingSQL is Now Open Source

#52

Post by shaklee3 »

esistgut wrote: post #44
felipe_aramburu wrote: post #41

Earlier quoted context omitted.

Nvidia loves and cherishes you (I think I don't work there). They want you to be able to do this on your laptop, your server, your super computer. If it has been a few years I would encourage you to get your feet wet again because support has gotten alot better. It's not like 5 years ago when it was nigh impossible to get the driver installed and weird conflicts would come up. I generally recommend using the debian i…

Do you use Nvidia products on Linux? Reading "love" and "Nvidia" in the same sentence feels a little bit odd because the general sentiment for Nvidia on the Linux community is "don't touch it with a 10 foot pole". If I remember correctly Torvalds himself named it the worst hardware company they had to deal with.

I'm not sure what you're talking about. Besides games, using CUDA on Linux has been the de facto OS for anything serious for almost as long as CUDA has existed. What exactly is the problem with it?

Re: BlazingSQL is Now Open Source

#53

Post by shaklee3 »

tombert wrote: post #20
felipe_aramburu wrote: post #17

Earlier quoted context omitted.

Thats a great question. The answer is two-fold. Early on when we first started playing around with General Processing on GPU's we had Nvidia cards to begin with and I started looking at the apis that were available to me. The CUDA ones were easier for me to get started, had tons of learning content that Nvidia provided, and were more performant on the cards that I had at the time compared to other options. So we buil…

Thanks for answering my question so quickly! That seems like a pretty good reason...I have been looking to learn some GPU programming to optimize some matrix math that I've been doing for a pet project, and while my first instinct was telling me OpenCL since it's portable, if people who actually know what they're talking about are saying that CUDA is simpler to start with, it might be worth it to me to pick up a chea…

You definitely do not want to use opencl for matrix multiplies on Nvidia cards. That's the most highly optimized task on GPUs, so much so that they have dedicated hardware units for it. Opencl cannot take advantage of those.

Re: BlazingSQL is Now Open Source

#54

Post by felipe_aramburu »

craftyguy wrote: post #46
felipe_aramburu wrote: post #17

Earlier quoted context omitted.

Thats a great question. The answer is two-fold. Early on when we first started playing around with General Processing on GPU's we had Nvidia cards to begin with and I started looking at the apis that were available to me. The CUDA ones were easier for me to get started, had tons of learning content that Nvidia provided, and were more performant on the cards that I had at the time compared to other options. So we buil…

> blazingsql is part of a greater ecosystem But now blazingsql is part of an ecosystem within a walled garden fully dependent upon the stability of a single company.

Well it pretty much always was a part of the eco system it just was not open source. We have been contributors to rapids for a while. And yes, we are betting on Nvidia for sure.

For most people building GP GPU solutions they are going to have to make a decision when it comes to which hardware they want to support. After that decision is made it really isn't something you can revisit without copious amounts of money.

Re: BlazingSQL is Now Open Source

#55

Post by felipe_aramburu »

chrisjc wrote: post #18
felipe_aramburu wrote: post #13

Earlier quoted context omitted.

This is a Distributed SQL engine not a database. We store no data. You store your data in HDFS, S3, posix, NFS etc. We allow you to query directly from these filesystems of the file formats you have already. You can look here to see the file formats cudf supports. https://github.com/rapidsai/cudf/tree/branch-0.9/cpp/src/io You can try it out yourself here https://colab.research.google.com/drive/1r7S15Ie33yRw8cmET7_..…

Is there any reason why a SQL format isn't is that list? Wondering if there's a way to join SQL sources with file storage sources. An example of this would be filtering or enrichment operations. // sorry if this is a stupid question.

When you say SQL format do you mean being able to read the output of a jdbc or odbc driver? If this is the case then mostly just time. You are not the first person to ask about this and now that there are java bindings in cudf this might become easier to make a reality in the next few months.

Or do you mean being able to read a database's file format natively? If this is the case there are many reasons. 1. There are many poorly/non documented formats 2. Even if you decide to read some other DB's format natively, those formats change over time 3. Little control of how and where the data is laid out

Re: BlazingSQL is Now Open Source

#56

Post by Xelbair »

dikei wrote: post #45
esistgut wrote: post #44

Earlier quoted context omitted.

Do you use Nvidia products on Linux? Reading "love" and "Nvidia" in the same sentence feels a little bit odd because the general sentiment for Nvidia on the Linux community is "don't touch it with a 10 foot pole". If I remember correctly Torvalds himself named it the worst hardware company they had to deal with.

Linus Torvalds's kernel developer point of view might be very different from the majority of users'. For the end users, they just need to install Nvidia's proprietary drivers and everything just works. For a long time, Nvidia was the best option for 3D graphics on Linux. ATI/AMD had terrible drivers (fglrx/Catalyst), Intel had abysmal performance.

>For the end users, they just need to install Nvidia's proprietary drivers and everything just works.

And that's the crux of the issue. proprietary drivers.

Re: BlazingSQL is Now Open Source

#57

Post by pjmlp »

Xelbair wrote: post #56
dikei wrote: post #45

Earlier quoted context omitted.

Linus Torvalds's kernel developer point of view might be very different from the majority of users'. For the end users, they just need to install Nvidia's proprietary drivers and everything just works. For a long time, Nvidia was the best option for 3D graphics on Linux. ATI/AMD had terrible drivers (fglrx/Catalyst), Intel had abysmal performance.

>For the end users, they just need to install Nvidia's proprietary drivers and everything just works. And that's the crux of the issue. proprietary drivers.

Hollywood and other 3D heavy creation studios are pretty fine with it.

Re: BlazingSQL is Now Open Source

#58

Post by jstimpfle »

esistgut wrote: post #44
felipe_aramburu wrote: post #41

Earlier quoted context omitted.

Nvidia loves and cherishes you (I think I don't work there). They want you to be able to do this on your laptop, your server, your super computer. If it has been a few years I would encourage you to get your feet wet again because support has gotten alot better. It's not like 5 years ago when it was nigh impossible to get the driver installed and weird conflicts would come up. I generally recommend using the debian i…

Do you use Nvidia products on Linux? Reading "love" and "Nvidia" in the same sentence feels a little bit odd because the general sentiment for Nvidia on the Linux community is "don't touch it with a 10 foot pole". If I remember correctly Torvalds himself named it the worst hardware company they had to deal with.

Well known video: https://www.youtube.com/watch?v=iYWzMvlj2RQ

Re: BlazingSQL is Now Open Source

#59

Post by orbifold »

esistgut wrote: post #44
felipe_aramburu wrote: post #41

Earlier quoted context omitted.

Nvidia loves and cherishes you (I think I don't work there). They want you to be able to do this on your laptop, your server, your super computer. If it has been a few years I would encourage you to get your feet wet again because support has gotten alot better. It's not like 5 years ago when it was nigh impossible to get the driver installed and weird conflicts would come up. I generally recommend using the debian i…

Do you use Nvidia products on Linux? Reading "love" and "Nvidia" in the same sentence feels a little bit odd because the general sentiment for Nvidia on the Linux community is "don't touch it with a 10 foot pole". If I remember correctly Torvalds himself named it the worst hardware company they had to deal with.

I think this sentiment exists solely among people that don’t actually own any NVIDIA hardware. I‘ve never had any problems with their drivers, any crashes in video games can be usually be attributed to be at least in part the Games fault. In contrast to Windows Linux has abysmal support for restarting crashed video drivers.

Re: BlazingSQL is Now Open Source

#60

Post by orbifold »

Xelbair wrote: post #56
dikei wrote: post #45

Earlier quoted context omitted.

Linus Torvalds's kernel developer point of view might be very different from the majority of users'. For the end users, they just need to install Nvidia's proprietary drivers and everything just works. For a long time, Nvidia was the best option for 3D graphics on Linux. ATI/AMD had terrible drivers (fglrx/Catalyst), Intel had abysmal performance.

>For the end users, they just need to install Nvidia's proprietary drivers and everything just works. And that's the crux of the issue. proprietary drivers.

The proprietary drivers are pretty nice and performant and have been for a long time. The same can’t be said about Intel (they don’t produce comparable hardware) or AMD (until recently their drivers were garbage, at the moment their best graphics card is worse than the best NVIDIA one)
Post reply on HN