Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres on the GPU

wiki.postgresql.org

Post reply on HN
Print view View original

11–20 of 79 posts

Re: Postgres on the GPU

#11

Post by smosher »

fixxer wrote: post #8
hippich wrote: post #2

NVidia's CUDA only (for now?) Anyone can explain why opensource projects embrace CUDA over OpenCL? As I understand OpenCL is more generic API which could be potentially used with CPUs and GPUs.

You're preaching to the choir. Looks like NEC funded this. NVidia seems to be the preferred hardware for institutions/big companies. I'm not sure if this is because NVidia's architecture is better for supercomputers or if they're simply better at marketing to those types of customers NVidia funds a lot of academics in my space, and I've found academia to be very anti-open source for those reasons, which amuses me gre…

NVidia funds a lot of academics in my space, and I've found academia to be very anti-open source for those reasons, which amuses me greatly.

Wait, are you saying academia is anti-opensource because of nVidia funding?

Re: Postgres on the GPU

#12

Post by smosher »

dljsjr wrote: post #7
hippich wrote: post #2

NVidia's CUDA only (for now?) Anyone can explain why opensource projects embrace CUDA over OpenCL? As I understand OpenCL is more generic API which could be potentially used with CPUs and GPUs.

There are probably a lot of factors. I worked on CUDA code for around a year, and used to understand the landscape pretty well, but if I were to start a high-performance computing project today I'd probably take my lumps and go with OpenCL. There would be a lot of lumps. Firstly, CUDA is just more mature; there is a very large and well-established set of libraries for a lot of common operations, there is a decent siz…

So why would you go with OpenCL? Is it portability? For a while I thought that was worth it but I am having a hard time remaining convinced of that.

Re: Postgres on the GPU

#13

Post by tkahn6 »

hippich wrote: post #2

NVidia's CUDA only (for now?) Anyone can explain why opensource projects embrace CUDA over OpenCL? As I understand OpenCL is more generic API which could be potentially used with CPUs and GPUs.

From my experience porting CUDA code to OpenCL code, CUDA is much cleaner and more succinct since it is able to assume a lot about the underlying hardware.

Re: Postgres on the GPU

#14

Post by cs702 »

If I understood correctly, this module allows you to export data to, and then access (read-only), foreign tables architected specifically for faster copying of data to/from GPU, allowing you to speed up queries that benefit from GPU computation by 10-20x. Nice.

On the surface, this seems very similar to https://news.ycombinator.com/item?id=5592886 except it's nicely integrated with Postgres. Or am I missing something?

tmostak?

Re: Postgres on the GPU

#15

Post by fixxer »

smosher wrote: post #11
fixxer wrote: post #8

Earlier quoted context omitted.

You're preaching to the choir. Looks like NEC funded this. NVidia seems to be the preferred hardware for institutions/big companies. I'm not sure if this is because NVidia's architecture is better for supercomputers or if they're simply better at marketing to those types of customers NVidia funds a lot of academics in my space, and I've found academia to be very anti-open source for those reasons, which amuses me gre…

NVidia funds a lot of academics in my space, and I've found academia to be very anti-open source for those reasons, which amuses me greatly. Wait, are you saying academia is anti-opensource because of nVidia funding?

I'm saying there is a systematic advantage to using proprietary technologies in academic research (companies have money, so you can write a grant and they will pay you $). Case in point, look at apps coming out of academia and you'll see a lot of WindowsPhone. It is because Microsoft gives away a ton of free phones (I have one on my desk at this moment) and Azure time.

Re: Postgres on the GPU

#16

Post by guard-of-terra »

hippich wrote: post #2

NVidia's CUDA only (for now?) Anyone can explain why opensource projects embrace CUDA over OpenCL? As I understand OpenCL is more generic API which could be potentially used with CPUs and GPUs.

Another question, is CUDA actually fast enough?

Because in bitcoin mining, it's always ATI/AMD and OpenCL because ATI cards are like ten times faster than Nvidia cards. This is because of architecture differencies.

Does it not affect this postgres table-scanning task? I wonder if they did any benchmarks.

Re: Postgres on the GPU

#17

Post by rarrrrrr »

Brilliant!

Perhaps it's just my scars showing, but I'm concerned about database system stability with active GPU hardware added to the box.

I probably wouldn't add the GPU hardware to a master but rather do the queries that would benefit from it on a streaming replica, where an occasional kernel panic won't be so severe.

Regardless, looking forward to trying it.

Re: Postgres on the GPU

#18

Post by cronin101 »

smosher wrote: post #12
dljsjr wrote: post #7

Earlier quoted context omitted.

There are probably a lot of factors. I worked on CUDA code for around a year, and used to understand the landscape pretty well, but if I were to start a high-performance computing project today I'd probably take my lumps and go with OpenCL. There would be a lot of lumps. Firstly, CUDA is just more mature; there is a very large and well-established set of libraries for a lot of common operations, there is a decent siz…

So why would you go with OpenCL? Is it portability? For a while I thought that was worth it but I am having a hard time remaining convinced of that.

I've been working on a rather large computation library using OpenCL. OpenCL is useful for providing an abstraction over multiple device types. If you are only interested in producing highly-tuned parallel code to execute on NVidia hardware, I suggest sticking to CUDA for the above reasons.

I utilised the OpenCL programming interface to write code that would run the same kernel functions on CPU and/or GPU devices (using heuristics to trade-off latency/throughput) which is something that is not possible afaik using the CUDA toolchain.

TL;DR YMMV and horses for courses.

Re: Postgres on the GPU

#19

Post by masklinn »

cs702 wrote: post #14

If I understood correctly, this module allows you to export data to, and then access (read-only), foreign tables architected specifically for faster copying of data to/from GPU, allowing you to speed up queries that benefit from GPU computation by 10-20x. Nice. On the surface, this seems very similar to https://news.ycombinator.com/item?id=5592886 except it's nicely integrated with Postgres. Or am I missing something…

> this module allows you to export data to, and then access (read-only), foreign tables architected specifically for faster copying of data to/from GPU

That seems to be pretty much it, although the read-only part is a limitation of Postgres rather than a feature of the module.

Furthermore, it seems to be dispatching queries intelligently so you can perform all queries against the FDW table with (I hope) minimal overhead, if the qualifier can't be compiled to a kernel the fdw will run it as a normal on-CPU qualifier. That's a thoughtful touch.

Re: Postgres on the GPU

#20

Post by fixxer »

mjn wrote: post #9
fixxer wrote: post #8

Earlier quoted context omitted.

You're preaching to the choir. Looks like NEC funded this. NVidia seems to be the preferred hardware for institutions/big companies. I'm not sure if this is because NVidia's architecture is better for supercomputers or if they're simply better at marketing to those types of customers NVidia funds a lot of academics in my space, and I've found academia to be very anti-open source for those reasons, which amuses me gre…

Matlab seems like inertia/culture to me: it's the longtime de-facto standard in engineering. Since it's what everyone uses, it's got packages for everything, and papers will often come with prototype Matlab implementations. Roughly like the cultural position R holds in statistics. Matlab's hold on engineering is also bolstered by its widespread use in industry: students want to learn it, because it's what their futur…

I've heard this argument a lot (since 1999), but I'm not confident it holds true anymore. The free alternatives are so good.

You may be right regarding Professors, but that is also changing as they age out.

Post reply on HN