Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPU Puzzles

github.com

Post reply on HN
Print view View original

21–30 of 44 posts

Re: GPU Puzzles

#21

Post by srush »

bytepoet wrote: post #7
srush wrote: post #4

I made these a couple of years ago as a teaching exercise for https://minitorch.github.io/ . At the time the resources for doing anything on GPUs were pretty sparse and the NVidia docs were quite challenging. These days there are great resources for going deep on this topic. The CUDA-mode org is particularly great, both their video series and PMPP reading groups.

Thanks a lot, Sasha, for creating these. I found your LLM training puzzles to be excellent as well.

Awesome! Here are all of them if anyone else is looking.

https://github.com/srush/Triton-puzzles https://github.com/srush/tensor-puzzles https://github.com/srush/autodiff-puzzles https://github.com/srush/transformer-puzzles https://github.com/srush/GPTworld https://github.com/srush/LLM-Training-Puzzles

Re: GPU Puzzles

#22

Post by srush »

xandrius wrote: post #11
fifilura wrote: post #5

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Astigmatism Maybe?

This: https://github.com/srush/GPU-Puzzles/raw/main/GPU_puzzlers_f...

Yeah it looks bad in the readme. In the actual code it's cleaner. Font rendering is hard

Re: GPU Puzzles

#23

Post by talldayo »

867-5309 wrote: post #15

seems like an opportune moment to gift a plug for bitcoin puzzles, namely BTC32 / 1000 BTC Challenge[1] pools are in dire need of cuda developers [1] https://bitcointalk.org/index.php?topic=1306983.0

> pools are in dire need of cuda developers

Pools have money; if they need CUDA engineers, they are fully capable of hiring them at the industry rate.

Re: GPU Puzzles

#25

Post by wmil »

So I'm used to working with lists and maps, which doesn't really track well with tackling problems on thousands of cores.

Is the usual strategy to worry less about repeating calculations and just use brute force to tackle the problem?

Is there a good resource to read about how to tackle problems in an extremely parallel way?

Re: GPU Puzzles

#26

Post by 867-5309 »

talldayo wrote: post #23
867-5309 wrote: post #15

seems like an opportune moment to gift a plug for bitcoin puzzles, namely BTC32 / 1000 BTC Challenge[1] pools are in dire need of cuda developers [1] https://bitcointalk.org/index.php?topic=1306983.0

> pools are in dire need of cuda developers Pools have money; if they need CUDA engineers, they are fully capable of hiring them at the industry rate.

most are community-based, plus, the prize can far exceed such a rate

Re: GPU Puzzles

#27

Post by lins1909 »

srush wrote: post #20
lins1909 wrote: post #18

Earlier quoted context omitted.

Thanks Sasha - this looks like a great resource.Just to be clear, would you recommend going through other newer resources than this instead? Not sure if your comment is to discourage someone from going through this.

These still hold up, and I think they're a great first step. But they no longer get you to the goal line. Think about it more as conceptual practice, before you enter the jungle.

Got it, thank you.

Re: GPU Puzzles

#28

Post by talldayo »

867-5309 wrote: post #26
talldayo wrote: post #23

Earlier quoted context omitted.

> pools are in dire need of cuda developers Pools have money; if they need CUDA engineers, they are fully capable of hiring them at the industry rate.

most are community-based, plus, the prize can far exceed such a rate

> the prize can far exceed such a rate

For all the good it's done them.

Re: GPU Puzzles

#29

Post by jamilton »

867-5309 wrote: post #15

seems like an opportune moment to gift a plug for bitcoin puzzles, namely BTC32 / 1000 BTC Challenge[1] pools are in dire need of cuda developers [1] https://bitcointalk.org/index.php?topic=1306983.0

Why? Wouldn't existing tools be about as good as they could be?

Re: GPU Puzzles

#30

Post by nextos »

srush wrote: post #16
nextos wrote: post #12

Earlier quoted context omitted.

Slightly offtopic, but any chance you could update or re-upload code for your https://github.com/harvardnlp/DeepLatentNLP tutorial? I found the NLP latent variable models discussed there really interesting, and notebooks were excellent. However, these seem gone and the only thing left are slides? Alternatively, any other places that discuss the same topics, including some code? I could only find equivalent discussion…

I'll take a look. Yeah Pyro is the best thing to do here. But it would be nice to revisit some of these implementationz

Thank you so much!
Post reply on HN