Reminds me of this post about how much of a gap there is between crypto mining and renting a box out to researchers:
https://news.ycombinator.com/item?id=16663689
We're way overdue for a p2p marketplace for cycles.
Building your own deep learning computer is 10x cheaper than AWS
241–250 of 269 posts
Re: Building your own deep learning computer is 10x cheaper than AWS
#242Earlier quoted context omitted.
According to the article you can break even after only two weeks which is far from "10k cookies every day for multiple years".
It's fine for exactly what the article describes: a student doing projects at home. For anything bigger than that, you'll quickly run into issues. Try talking to your ops team and telling them you want to set up a mid-range desktop PC with extra RAM and a high end graphics card as a DL workstation. I think you'll very quickly find some friction, especially once you want to go into production with it.
And that's just unnecessary friction, which is a problem of politics, not technology.
I do recall a time as recently as 4 years ago where it was difficult to find, for example, rack-mountable servers that could hold a large number of these GPU cards, but substituting for a mid-range PC with 2 or 4 of them would not have been an issue, even then.
Cost would be higher but not necessarily outrageously so, since large server chasses often have startlingly large PSUs. It might be an extra 50% on top of that $3k, but that's still competitive, even with the pre-builds the article mentions.
Re: Building your own deep learning computer is 10x cheaper than AWS
#243Great post. From someone who's done a few of these, I'll make a few observations: 1. Even if your DL model is running on GPUs, you'll run into things that are CPU bound. You'll even run into things that are not multithreaded and are CPU bound. It's valuable to get a CPU that has good single-core performance. 2. For DL applications, NVMe is overkill. Your models are not going to be able to saturate a SATA SSD, and wit…
In terms of number 6. The Linux Deep Learning framework installation can happen in one line with Lambda Stack:
Re: Building your own deep learning computer is 10x cheaper than AWS
#244I have a hard time taking seriously any analysis that states x is n-times smaller than y, with neither being a negative number. That's not how math works.
It is if you define cheapness is 1/price. So $10 is 0.1 cheap, and $100 is 0.01 cheap. So $10 is 10x cheaper than $100.
Re: Building your own deep learning computer is 10x cheaper than AWS
#245TLDR; Two month rental cost of GPU machine in AWS (and elsewhere) is same as purchase price of similarly powered hardware. Why? In fiersly competitive market no cloud provider wants to capture share here. One thing is that author is comparing high end GPU machine with V100s that cost way too much for the privilege of putting them in data center compared to consumer grade Ti1080.
Re: Building your own deep learning computer is 10x cheaper than AWS
#246Earlier quoted context omitted.
They seriously can't buy a graphics card and slap it in the PCIe slot?
You're neglecting management costs. IT teams don't buy hardware with corporate credit cards, they have to work through pre-existing requisition processes that properly budget for the hardware, make sure support contracts are in place, etc. You have to migrate whatever workload off the server where you installed the GPU (politically problematic since Murphy promises you that your users will be connecting to the server…
In most (good) companies team/group have budgets where large portions are at discretion to group manager. I would expect group managers to approve for such hardware requests. It’s much harder to justify $4K bills on cloud services. This is not IT issue and I think this thread has got derailed little here.
Re: Building your own deep learning computer is 10x cheaper than AWS
#247You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)
>>politely tell them it's their own ... fault when they complain about the costs.
They don't care. Neither do their bosses. Not even the CFO or the CEO.
The only people who will care are the investors. And sometimes not even them, they will likely just sell and walk.
Only people who are likely to care are activist investors with large block of shares, those types have vested interests in these things.
Part of the reason why there is so much waste every where is because organizations are not humans. And most people in authority have no real stake or long term consequences for decisions. This is everywhere, religious organizations, companies, governments etc. Everywhere.
Re: Building your own deep learning computer is 10x cheaper than AWS
#248You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)
We just got around them and argue after the fact. It is a pain no matter what. But we actually do both, interernal custom builds and cloud computing so no matter what we are set
Re: Building your own deep learning computer is 10x cheaper than AWS
#249Earlier quoted context omitted.
If you buy equipment you can only deduct its depreciation costs every year. If you pay for services you can deduct the full cost from your taxes.
Anyone who thinks spending $10 instead of $1, so that you can book the expense (presumably for a tax write-off which might save you 30%... MAYBE?) needs to stay away from Finances.
Correct, unless the org doesn't have the money to make all the capex upfront
Re: Building your own deep learning computer is 10x cheaper than AWS
#250Earlier quoted context omitted.
In my experience, the blind drive to the cloud comes from above. It's someone's pet project and you're either on-board or not. I won't go as far as calling it yet another cyclical IT phase, but it has all the hallmarks of one.
There are armies of salespeople brainwashing executives about "the cloud". At my last job it only took a few years before every single C-level was spouting off recycled cloud propaganda
That's ironic-- back in 2014, every prospect was hanging up on me saying they "weren't ready to go cloud yet" :)