Build a fast deep learning machine for under $1K
1–10 of 65 posts
Re: Build a fast deep learning machine for under $1K
#2If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board.
Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better.
Choose your video card wisely, AMD does some things better than NVIDIA depending on what software you're running.
If you chose an ITX platform for deep learning fun, uh, you should really add the couple shoeboxes extra space to your platform and have 4x-7x the power available to upgrade into.
Re: Build a fast deep learning machine for under $1K
#3Contrary to the claims there, the CPU does matter: if the network is relatively small, it's easy to get bottle-necked feeding the GPU.
Same thing applies to storage, for the same reasons. There's a reason AMD is selling deep learning cards with Flash drives built in.
The choice of CPU cooler is silly. The CPU chosen is a 65W model, which will be cooled quietly even by the boxed cooler (and we don't need to care about overclocking with a non-K). By his own admission, the rig draws 250W, so the idea that 650W must be cooled is nonsense, and even then, only a part of that is due to the CPU.
It's probably worth looking more closely at the GPU, notably, if that cooler stays quiet when the card is under full load.
You do not need a monitor, keyboard or mouse to use such a machine. You can just ssh into it. You might need to borrow a set to set up the BIOS.
If you "run out of memory for your applications" on the GPU, have you considered simply lowering the mini-batch size? (See, this is why we got a good CPU, it removes the hurt from feeding smaller batches)
Re: Build a fast deep learning machine for under $1K
#4If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board. Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better. Choose your video card wisely, AMD does some thi…
Try to bring good in the conversation.
What piece you would have use? Why? What tradeoffs you have considered?
You sound expert in the field, try to share your knowledge with the community in a constructive way so that we can all benefit from it.
I develop software for living and still I haven't any clue of what you said, while I followed quite well the article and I thought he was making reasonable choice.
Please show me better.
Re: Build a fast deep learning machine for under $1K
#5If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board. Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better. Choose your video card wisely, AMD does some thi…
Try to bring good in the conversation. What piece you would have use? Why? What tradeoffs you have considered? You sound expert in the field, try to share your knowledge with the community in a constructive way so that we can all benefit from it. I develop software for living and still I haven't any clue of what you said, while I followed quite well the article and I thought he was making reasonable choice. Please sh…
I'd like to second this request and note it took me years to figure out that people saying "focus on the positive" in this context would be satisfied by me phrasing things in terms of what to do rather than what not to do. Concrete examples are key for this approach.
Re: Build a fast deep learning machine for under $1K
#6My home server has an i7 4790K, 32GB DDR3 and... a GTX 1080 Gamer Edition that's doing... well, bugger all. Why is the graphics card in there you might ask? Well, it was sitting on my desk collecting dust as I wait patiently for a Pascal driver for eGPUs on macOS but Nvidia had delayed these for long enough that I thought I'd chuck it in my home server, it is being used by ffmpeg for x265/HVEC transcoding with Plex media server but I'm really not even touching it's capabilities - not even close when transcoding a 120GB+ 4K 120FPS video and streaming it wirelessly to my TV that's running a Plex client. The server is running CentOS 7 and has 10x 8TB SATA drives in BTRFS RAID 10 for storage, two Intel NVMe 750 series 1.2 TB SSDs for the OS as well as a couple of 2TB Crucial MX300 SATA SSDs that aren't currently in use for much.
So, I have been sitting here wondering how I can make better use of it while it's practically parked in my server while I wait for some Pascal drivers to drop. I'm not so interested in robots / live video processing but it'd be neat if I could leverage its power for something fun or to assist the servers compute performance in some other ways that might be useful or interesting. I was running KVM with PCIe pass through passing the GPU to a guest VM running Steam as a steam streaming box but I got sick of having an OS on the network that I had to worry about Virus's, Malware and annoyingly slow and unreliable updates being installed when you least want it to happen and I got lazy and went back to playing PlayStation when I needed some time out from constructive work / research.
Open to ideas to any software I could tinker with if anyone has any interesting suggestions?
Re: Build a fast deep learning machine for under $1K
#7If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board. Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better. Choose your video card wisely, AMD does some thi…
> Choose your video card wisely, AMD does some things better than NVIDIA depending on what software you're running.
Deep learning toolkits basically have two modes of operation: the CPU way, and the proprietary NVIDIA way. There is no point to putting an AMD chip in a machine you hope to use for deep learning.
Some toolkits may go through the motions of supporting AMD via OpenCL, but that's not going to be the case that they make sure works well, or works at all.
This is a problem, as NVIDIA is awful at maintaining drivers, results are being published based on "well, my NVIDIA black box decided to do this", and it contributes to deep learning veering toward a local maximum. But specifically choosing to do deep learning with an AMD chip is a pointless sacrifice.
Re: Build a fast deep learning machine for under $1K
#8In Europe it's also very common to just choose the components in a web shop and have them assembled, also for about $1000. (Not looking at Dell)
Anyone interested in deep learning should go that path instead of burning their money on Amazon.
Re: Build a fast deep learning machine for under $1K
#9In Europe it's also very common to just choose the components in a web shop and have them assembled, also for about $1000. (Not looking at Dell) Anyone interested in deep learning should go that path instead of burning their money on Amazon.
+1 on that. There are a number of online build-to-order shops here in the UK which offer a truly vast range of component combinations. Better still, the order process automatically checks for compatibility with, for example, your choice of motherboard.
Once you have the configuration you want, you just pay up and it arrives professionally assembled ready to rock. I don't think I'll ever by an off-the-shelf machine again.
Re: Build a fast deep learning machine for under $1K
#10If by "super fast deep learning machine" you actually mean, mid tier gaming computer. Uh, sure. Don't spend more on an ITX board, when you can fit 4x to 7x the video cards in a full size ATX computer using pci-e risers, while paying less for the board. Don't lose 5 percent of your cpu overclock or more by having an itx friendly cooler instead of a D15 noctua or better. Choose your video card wisely, AMD does some thi…
Try to bring good in the conversation. What piece you would have use? Why? What tradeoffs you have considered? You sound expert in the field, try to share your knowledge with the community in a constructive way so that we can all benefit from it. I develop software for living and still I haven't any clue of what you said, while I followed quite well the article and I thought he was making reasonable choice. Please sh…
Not parent, but I've built my own computers for almost 20 years.
As the articles author himself questions, he should have gotten the marginally more expensive CPU, and definitely the GPU with more RAM.
* The overclockable CPU doesn't just mean that you very easily could get a 10%+ performance boost without much work, but also that you often (depending on your specific chip) can lower the voltage and make it run much cooler/quieter, which is something I increasingly care about if I'm using it a lot.
* He writes that fitting the model in RAM is basically the most important part, but then saves less than a dinner out by basically halving the RAM on the GPU he bought. The chip is otherwise the same though, so performance is only dependent on whether you fill that RAM or not.
* The Noctua D15 the parent mentioned is a CPU cooler where Noctua is a long standing high performant brand, and D15 is a specific model with a 15 inch fan and comparable sized heat sink. There are of course other brands, but I myself also usually end up with Noctuas. The reason it's important is that however fast you can dissipate heat from the cpu/case, the less chance of throttling, and the larger headroom for potential overclocks you get.
Airflow and room for larger heatsinks is also why he recommended not going for an ITX. A linked benefit is again the potential for a quieter system.
I haven't gone much into ML (yet), but I currently have a system with:
* I7 6700K (the difference to I5 6600K being higher base clock and hyperthreading, which is more important to computational work than to gaming, so if you have the money, definitely go for the I7)
* 32GB DDR4 (as author mentioned, RAM is cheap). The clock/timings on RAM isn't really as important, but try to find the best you can find for a given price point.
* An Nvidia GTX 1080: It's not Titan X or Z, but almost, at less price. It definitely blows the budget for a $1000 system, but I agree that the entire 10 series is good.
If the limit is a firm $1000, I would get something like this:
https://pcpartpicker.com/list/XHV9Fd
And if more funds is available, I'd get more storage and RAM, then a better CPU, then a better GPU, then maybe bump the chassis up to an R5 (same brand), possibly another motherboard. In that order. There's always something better, so you compromise based on budget.