41–50 of 52 posts
Data truly is the new oil. When it’s all done the compute costs and code will be cheap or free. There’s a lot hinging on how we interpret copyright laws or what kind of data rights laws we enact.
Earlier quoted context omitted.
The companies that wait for a 100% "clean" model are going to get left behind. e.g. ChatGPT launching despite Google, Meta and others already having very similar technology internally
There is already a class action lawsuit. The companies that move forward with "dirty" models can be wiped out by legal fees before they got off the ground.
Earlier quoted context omitted.
Hi, one of the authors of the post, we will update the post with numbers from 256 GPU run within the next few days. We estimated the 256 run to be the fastest (13 days), but also the most expensive at $160k. The measured 128 GPU run would take 21 days but for $125k if you are interested in lower costs.
That is not necessarily a saving. If you have a let's say a team of five people each costing $1000 a day, those idle 6 days (not counting the weekend) would add up to $30k of wasted money. Then if you are working on something the competition is also working on, these lost days would add up and potentially cost losing the edge - could be quite expensive or even cost the business.
Interesting to think about where the cost will go in a few years. I remember in college intro to CS class back in 1998, where I heard the story of building the first computer that could perform at 1 TFLOPS[1]. It cost $46 million and took up 1600 square feet. Now a $600 Mac Mini will do double that. [1] https://en.wikipedia.org/wiki/ASCI_Red
Is this just an ad for a service? They didn’t make anything. This is just speculative benchmarking. I am deeply not interested in multiplying the numbers on your pricing sheet by the estimated numbers on the stable diffusion model card. I have zero interest in your (certainly excellent) Proprietary Special Sauce (TM) that makes spending money on your service a good idea. This just reads as spam that got past the spam…
> Try out our Stable Diffusion code here! > https://github.com/mosaicml/diffusion-benchmark
Earlier quoted context omitted.
Those models are not from scratch.
It is a fair point though - there's no utility in training an openly available model from scratch. Finetuning is far more practical.
Interesting to think about where the cost will go in a few years. I remember in college intro to CS class back in 1998, where I heard the story of building the first computer that could perform at 1 TFLOPS[1]. It cost $46 million and took up 1600 square feet. Now a $600 Mac Mini will do double that. [1] https://en.wikipedia.org/wiki/ASCI_Red
It is not going to go down much anymore, because the end of Moore's law has been reached as physical limitations become a factor. You cannot scale chips close to 1 atom wide transistors.
Earlier quoted context omitted.
It is not going to go down much anymore, because the end of Moore's law has been reached as physical limitations become a factor. You cannot scale chips close to 1 atom wide transistors.
I’ll take this 10 year bet. You really think nvidia is just gonna stop releasing new revisions? “Moores law is dead” is way over-memed, it’s more of an axiom about how computers continually improve than really being about transistor count at this point.
Nvidia's performance gains in recent years has been about scaling chip size and making more efficient use of each transistor both in terms of power and count than anything else. A large part of that is minimizing how far data physically moves for any given workloads via stuff like HBM, memory compression, and smarter/larger caches.
In fact, Nvidia doesn't even really try to be on the bleeding edge nodes anymore because per transistor costs has been trending up or level on bleeding edge nodes for at least 5 years now.