Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

leblancfg.com

Post reply on HN
Print view View original

31–40 of 42 posts

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#31

Post by leblancfg »

illumin8 wrote: post #3

So basically, your calculations for deciding whether purchasing physical GPUs or using cloud GPUs is more cost effective should add $5,000 (50 hours at $100 an hour seems reasonable) onto the cost of any physical rig, and assume you have a spare 50 hours of opportunity cost to set it up. All the ML researchers lusting for bare metal need to decide if they want to be "Gentoo Ricers" - https://funroll-loops.teurasporsa…

Author here. Good point, but 50 hours seems like an outlier for sure: I'm no expert whatsoever when it comes to debugging hardware in Linux. Assuming setup time across all first-time builds is Poisson-distributed, feels like the peak of that curve is more like 2-3 hours. Certainly something to keep in mind if considering doing it, though.

Thoughts:

* If I was setting up multiple identical machines, I'd only need to solve the issue once.

* Details matter: Switching from VGA to HDMI connector made a world of difference. Me not noticing that earlier cost me ~2 days of trial-and-error to go to waste.

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#32

Post by leblancfg »

solomatov wrote: post #24

The best way to setup tensorflow for gpu is to use nvidia-docker and pre-built docker images. They mention it in their documentation.

Author here. You would still need working NVIDIA drivers and CUDA to use nvidia-docker, which I couldn't manage to do.

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#33

Post by leblancfg »

nextos wrote: post #25

The only two distributions I've tried where this worked with a one liner were NixOS and Arch. I'm surprised this doesn't work in Manjaro given that it's an Arch derivative. But I don't see much point in Arch derivatives as they do away with Arch's selling point. No layers between you and upstream.

I see what you mean, and I agree with you in theory. The long learning curve and installation time for Arch was a non-starter for me though.

Back to your point, Manjaro runs directly on pacman repos, not a layer between it and upstream. It just comes with IMO sane defaults and an installer GUI.

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#34

Post by foepys »

leblancfg wrote: post #33
nextos wrote: post #25

The only two distributions I've tried where this worked with a one liner were NixOS and Arch. I'm surprised this doesn't work in Manjaro given that it's an Arch derivative. But I don't see much point in Arch derivatives as they do away with Arch's selling point. No layers between you and upstream.

I see what you mean, and I agree with you in theory. The long learning curve and installation time for Arch was a non-starter for me though. Back to your point, Manjaro runs directly on pacman repos, not a layer between it and upstream. It just comes with IMO sane defaults and an installer GUI.

But don't expect support from the Arch Linux forums. They really, really don't like Manjaro as it attracts people that shouldn't be running Arch in the first place. Arch Linux is not for beginners.

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#35

Post by nextos »

leblancfg wrote: post #33
nextos wrote: post #25

The only two distributions I've tried where this worked with a one liner were NixOS and Arch. I'm surprised this doesn't work in Manjaro given that it's an Arch derivative. But I don't see much point in Arch derivatives as they do away with Arch's selling point. No layers between you and upstream.

I see what you mean, and I agree with you in theory. The long learning curve and installation time for Arch was a non-starter for me though. Back to your point, Manjaro runs directly on pacman repos, not a layer between it and upstream. It just comes with IMO sane defaults and an installer GUI.

When did you try Arch? With the adoption of systemd it's incredibly easy to configure if you aim for a simple no desktop enviroment setup or something minimal.

If not, I think it doesn't make any sense to use it unless you are a seasoned user. Rolling release will make running big frameworks like KDE tricky, as things will change often. And unless you have set everything up yourself, you won't know where to look in order to fix things.

I'd recommend you look into NixOS if you want to run a desktop environment. It's radically different (totally functional and declarative, whereas Arch is imperative). All Python and deep learning stuff is neatly packaged and tested. And rollbacks are trivial. There's no state.

You can even run Nix in other distros or macOS.

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#36

Post by alfalfasprout »

mippie_moe wrote: post #21

My company (Lambda Labs) made it possible to install cuda, cudnn, tensorflow, pytorch, and other deep learning frameworks with one line: https://lambdalabs.com/lambda-stack-deep-learning-software We wrote debian packages for every framework, including cuda and cudnn. Using our Debian repository, you can install all these frameworks using apt/aptitude. When a new version of a framework comes out, we usually have it av…

Any chance you can get MxNet (and Keras MxNet) in there?

Installing this stuff is a huge nuisance for us and we have some pretty insane Dockerfiles to handle all the different combinations. I might look into using this for our ML images.

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#37

Post by vokep »

mippie_moe wrote: post #21

My company (Lambda Labs) made it possible to install cuda, cudnn, tensorflow, pytorch, and other deep learning frameworks with one line: https://lambdalabs.com/lambda-stack-deep-learning-software We wrote debian packages for every framework, including cuda and cudnn. Using our Debian repository, you can install all these frameworks using apt/aptitude. When a new version of a framework comes out, we usually have it av…

...And then you take all my passwords right? Or mine some btc quietly in the background?

Seriously this is too cool and nice, theres no way you're so cool and nice, such people just don't exist! (really this is awesome, thank you)

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#38

Post by ericd »

tebruno99 wrote: post #8

Yeah, not sure why you would be in Grub for Nvidia graphics drivers. This whole thing seems pretty odd to me since I've not seen a Nvidia driver problem in several years. Was he trying to use like Ubuntu 12.04 or something. Manjaro is a great choice and it does "just work" the best. All the others work just fine though.

I've seen some issues with some of the more recent Nvidia drivers+newer kernels which caused a login loop on Ubuntu 16.04.

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#39

Post by mippie_moe »

vokep wrote: post #37
mippie_moe wrote: post #21

My company (Lambda Labs) made it possible to install cuda, cudnn, tensorflow, pytorch, and other deep learning frameworks with one line: https://lambdalabs.com/lambda-stack-deep-learning-software We wrote debian packages for every framework, including cuda and cudnn. Using our Debian repository, you can install all these frameworks using apt/aptitude. When a new version of a framework comes out, we usually have it av…

...And then you take all my passwords right? Or mine some btc quietly in the background? Seriously this is too cool and nice, theres no way you're so cool and nice, such people just don't exist! (really this is awesome, thank you)

Thanks for the positive feedback!

Our company used to run a cluster of ~1000 gpu servers for inference and training. We used caffe and torch. Provisioning and software maintenance was a huge hassle.

We realized how painful it is to get a machine set up for deep learning, so we decided to release our debian packages to the public.

We sell computers built for deep learning researchers and figure the more people we can help get started, the better :)

Re: GPU Headaches: Notes on Installing CUDA, CuDNN and Tensorflow on Manjaro

#40

Post by mippie_moe »

alfalfasprout wrote: post #36
mippie_moe wrote: post #21

My company (Lambda Labs) made it possible to install cuda, cudnn, tensorflow, pytorch, and other deep learning frameworks with one line: https://lambdalabs.com/lambda-stack-deep-learning-software We wrote debian packages for every framework, including cuda and cudnn. Using our Debian repository, you can install all these frameworks using apt/aptitude. When a new version of a framework comes out, we usually have it av…

Any chance you can get MxNet (and Keras MxNet) in there? Installing this stuff is a huge nuisance for us and we have some pretty insane Dockerfiles to handle all the different combinations. I might look into using this for our ML images.

I’ll talk to the team!
Post reply on HN