131–140 of 269 posts
You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)
Earlier quoted context omitted.
So, what's the optimal solution according to the Ski Rental problem?
If you have no prior knowledge of how often you will be skiing, then the optimal solution is exactly 50% the cost of buying the skis upfront.
Earlier quoted context omitted.
Respectfully, those companies' cloud architects suck. If someone goes to the trouble to migrate onto cloud, and then replicates pre-devops workflows... wow.
Not every company is a single page webapp and simple service portfolio... I work at a place with 3,000+ developers and over 700 applications - there is no way in hell our cloud portfolio would have any standards if we didn't have a robust operations/engineering team making it work. Sometimes, even when you adopt the cloud, you realize that your operations model is even more important and there is nothing wrong with t…
Earlier quoted context omitted.
If you need 10k cookies every day for multiple years, you're better off building your own bakery. If you need multiple years of continuous DL work, you're better off building your own DL system. If you need 5 cookies today (and maybe the next batch next week), go to Walmart. Same for an one-off DL job.
According to the article you can break even after only two weeks which is far from "10k cookies every day for multiple years".
For anything bigger than that, you'll quickly run into issues. Try talking to your ops team and telling them you want to set up a mid-range desktop PC with extra RAM and a high end graphics card as a DL workstation. I think you'll very quickly find some friction, especially once you want to go into production with it.
Earlier quoted context omitted.
Not every company is a single page webapp and simple service portfolio... I work at a place with 3,000+ developers and over 700 applications - there is no way in hell our cloud portfolio would have any standards if we didn't have a robust operations/engineering team making it work. Sometimes, even when you adopt the cloud, you realize that your operations model is even more important and there is nothing wrong with t…
The parent's point was that if the ops/deployment engineering team is unresponsive to the needs of developers, it may end up being better to run with no standards in cowboy-mode. If they ops team is extremely fast and highly skilled, they will be a boon full stop. If they are unskilled and politically obstructive, they will be a curse, full stop.
While, in sheer dollar amount this post is probably correct, it doesn't really scale. At scale, you need more than just hardware. It's maintenance, racks, cooling, security, fire suppression etc. Oh, and the cost of replacing the GPUs when they die. At full price, yes, cloud GPUs on AWS aren't cheap, but at potentially a 90% saving in some regions/AZs, the price of spot instances by bidding on unused capacity for ML…
The equivalent of an i3 metal is probably around 30000 to 40000$ with Dell or HP, and probably half cheaper if self assembled (like a supermicro server). AWS i3.metal will cost 43000 annually, so even more than the acquisition cost of the server, server which will last probably around 5 year.
But if you start taking into account all the logistic, additional skills, people and processes needed to maintain a rack in a DC, plus the additional equipment (network gears, KVMs, etc). The cost win is far less evident and it also generally adds delays when product requirements changes.
Fronting the capital can be an issue for many companies, specially the smaller ones, and for the bigger ones, repurposing the hardware bought for a failed project/experiment is not always evident.
I continue to be boggled that Alex Krizhevsky's One Weird Trick never made it to TensorFlow or anywhere else:
https://arxiv.org/abs/1404.5997
I also suspect that's why so many thought leaders consider ImageNet to be solved, when what's really solved is ImageNet-1K. That leaves ~21K more outputs on the softmax of the output layer for ImageNet-22K, which to my knowledge, is still not solved. A 22,000-wide output sourced by a 4096-wide embedding is 90K+ parameters (which is almost 4x as many parameters in the entire ResNet-50 network).
All that said, while it will always be cheaper to buy your ML peeps $10K quad-GPU workstations and upgrade their consumer GPUs whenever a brand new shiny becomes available, be aware NVIDIA is very passive aggressive about this following some strange magical thinking that this is OK for academics, but not OK for business. My own biased take is it's the right solution for anyone doing research, and the cloud is the right solution for scaling it up for production. Silly me.
You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)
That's very passive aggressive way to deal with it. If that's your only option you are really a cubicle slave in corporate hell. In my opinion it's better to escalate upwards with proposals and not back down easily. You just have to frame it correctly and use right names and terms. * Usually big companies understand the concept of "a lab" that has infrastructure managed outside the corporate IT. Once you fight the ha…
THIS
> Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it.
It would be awesome. I also wonder if in this case it is an issue of hardware or only something related to the drivers/API.
Can they make something similar/backwards-compatible with Cuda but cheaper/better?
Earlier quoted context omitted.
I wondered how it is possible that they can restrict the use of hardware after it is bought. After reading the article I learned that they key is the license to the drivers. So I guess in theory it would be possible for AWS to develop their own (or enhance open source) drivers. On the other hand they would spoil the business relationship with Nvidia and have to do without any discounts.
The problem is that there's already open-source nvidia driver -- https://nouveau.freedesktop.org -- but it's so feature incomplete that it's good only for hobby. You can't run CUDA on it, you can't run any recent games on it, you can run TensorFlow on it, you can't run PyTorch on it, etc etc -- it is essentially useless and has been so for many years.