Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Every Developer Should Know About GPU Computing (2023)

blog.codingconfessions.com

Post reply on HN
Print view View original

41–46 of 46 posts

Re: What Every Developer Should Know About GPU Computing (2023)

#41

Post by hermitcrab »

TheDudeMan wrote: post #39
hermitcrab wrote: post #31

GPU are optimized for number crunching. Do they get used at all for string processing? I ask because I develop data wrangling software and most of it is string processing (joins, concatenations, aggregations, filtering etc), rather than numerical.

Do you have millions of strings that need to be manipulated in the same way at the same time?

Yes. For example, you might want to change a column of 10 million strings from upper case to lower case. Or concatenate 2 columns to create a third column. It is not clear to me this would be any faster on a GPU.

Re: What Every Developer Should Know About GPU Computing (2023)

#42

Post by cgreerrun »

penr0se wrote: post #7
cgreerrun wrote: post #2

This video is a great explainer too: How do Graphics Cards Work? Exploring GPU Architecture ( https://youtu.be/h9Z4oGN89MU?si=EPPO0kny-gN0zLeC )

Wow, this is one of the best videos I've ever watched. Thanks for sharing

You're welcome! I highly recommend the CPU/microchip one too.

Re: What Every Developer Should Know About GPU Computing (2023)

#43

Post by hermitcrab »

TheDudeMan wrote: post #39
hermitcrab wrote: post #31

GPU are optimized for number crunching. Do they get used at all for string processing? I ask because I develop data wrangling software and most of it is string processing (joins, concatenations, aggregations, filtering etc), rather than numerical.

Do you have millions of strings that need to be manipulated in the same way at the same time?

Also, you might want to create a hash table from a million values in a column, so you can use this for a join.

Re: What Every Developer Should Know About GPU Computing (2023)

#44

Post by GregarianChild »

adrian_b wrote: post #30
zokier wrote: post #23

Earlier quoted context omitted.

While I do understand conceptually that GPU is basically its own computer, I struggle to understand how this works in terms of operating systems and multitasking. Fundamentally managing resources between tasks is one of the core functions of operating systems, and stuff like CPU schedulers and virtual memory are fairly well understood. But how are the resources on GPUs managed? If I have n processes doing GPU compute…

Scheduling and allocation are done by the GPU driver, e.g. the CUDA runtime, with some hardware/firmware assistance from the GPU, which also contains one or more microcontrollers which may perform some of the tasks required for this.

Can you point me to reading material about how the CUDA runtime does this with hardware assistance? I looked but I have been unable to find any thing persuasive in this direction.

Re: What Every Developer Should Know About GPU Computing (2023)

#45

Post by ElFitz »

b3orn wrote: post #24
ElFitz wrote: post #16

Earlier quoted context omitted.

> Another kind of misconception: data transfer is a _really_ overlooked issue. […] If you want to write 20mb of data to a buffer, that's not just a memcpy, all that data has to go over the PCIe buss to the GPU […], and that's going to be expensive (in real time contexts). Similarly if you want to read a whole large buffer of results back from the GPU, that's going to take some time. Does having a unified memory, like…

In theory yes, because you wouldn't need to copy the data, in practice it depends on the API and you might end up copying data from RAM to RAM. If the API doesn't allow you to simply pass an address to the GPU then you need to allocate memory on the GPU and copy your data to that memory, even if it's unified memory.

Good to know, thanks!

Re: What Every Developer Should Know About GPU Computing (2023)

#46

Post by winwang »

etdeagle2 wrote: post #5
winwang wrote: post #4

Makes me consider writing a post on misconceptions of GPU computing, such as requiring the problem to be fully data-parallel.

Please do! I would love to read about it. I have been playing with GPU hash tables shared between all the threads using locking (Interlocked.CompareExchange) and such in a Compute Shader. I have been wondering if there are better ways than locking.

Until I get around to it, there's this blog post I wrote: https://wiwa.substack.com/p/can-we-10x-rust-hashmap-throughp...

Not an "expert" on GPU hash tables, but I'm mildly aware of other hashmap things like a novel(?) hash-indexed sorted array (HISA): https://arxiv.org/html/2311.02206v3

Post reply on HN