Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are GPUs Worth It for ML?

exafunction.com

Post reply on HN
Print view View original

91–100 of 100 posts

Re: Are GPUs Worth It for ML?

#91

Post by einpoklum »

fomine3 wrote: post #77
einpoklum wrote: post #51

> And CPUs are so much cheaper Doesn't look like it. Consumer: AMD ThreadRipper 3970X: ~3000 USD on NewEgg https://www.newegg.com/amd-ryzen-threadripper-2990wx/p/N82E1... NVIDIA RTX 3080 Ti Founders' Edition: ~2000 USD https://www.newegg.com/nvidia-900-1g133-2518-000/p/1FT-0004-... For servers, a comparison is even more complicated and it wouldn't be fair to just give two numbers, but I still don't think GPUs are mor…

Consumer GPUs are very cheap but prohibited to use it on datacenter.

In a datacenter you need to compare Xeon's and Epyc's with Telsa's.

Re: Are GPUs Worth It for ML?

#93

Post by macksd »

fragmede wrote: post #88
macksd wrote: post #72

Earlier quoted context omitted.

Isn't a key selling point of the latest, hottest model that's on the front page of Hacker News multiple times right now, the fact that it fits on consumer-grade GPUs? Surely some of the interesting ideas it's spawning right now are people doing transfer learning on GPUs that don't end in "100", don't you think?

for what it's worth, stable diffusion was trained on 32 x 8 x A100 GPUs

You know there's a huge difference between training the original model and transfer learning to apply it to a new use case, right? Saying people are years behind if they think there work is only worth something with 8 A100 pods is pretty ignorant of how most applications get built. Not everyone's trying to design novel model architectures, nor should they.

Re: Are GPUs Worth It for ML?

#95

Post by wmf »

jgauth wrote: post #94
wmf wrote: post #67

Earlier quoted context omitted.

Because that's "illegal" so cloud providers can't do it.

Can you describe what you mean by that?

The GeForce driver EULA doesn't allow it to be used in servers or something like that, so clouds all have to use the more expensive professional cards.

Re: Are GPUs Worth It for ML?

#96

Post by [deleted] »

MichaelBurge wrote: post #34
PeterisP wrote: post #7

For some reason they focus on the inference, which is the computationally cheap part. If you're working on ML (as opposed to deploying someone else's ML) then almost all of your workload is training, not inference.

Think Google: Every time you search, some model somewhere gets invoked, and the aggregate inference cost would dwarf even very large training costs if you have billions of searches. Marketing blogspam like this is always targeting big(not Google, but big) companies hoping to divert their big IT budgets to their coffers: "You have X million queries to your model every day. Imagine if we billed you per-request, but sca…

[deleted]

Re: Are GPUs Worth It for ML?

#97

Post by atq2119 »

jimmygrapes wrote: post #78

Perhaps it's been mentioned before but I do find it curious how often crypto mining was lambasted for contributing to climate change get I haven't seen anybody bat an eye at a fairly similar amount of compute power used for ML applications. Makes me wonder.

The two are quite different when you look at the cost/benefit ratio.

Re: Are GPUs Worth It for ML?

#98

Post by atq2119 »

fomine3 wrote: post #77
einpoklum wrote: post #51

> And CPUs are so much cheaper Doesn't look like it. Consumer: AMD ThreadRipper 3970X: ~3000 USD on NewEgg https://www.newegg.com/amd-ryzen-threadripper-2990wx/p/N82E1... NVIDIA RTX 3080 Ti Founders' Edition: ~2000 USD https://www.newegg.com/nvidia-900-1g133-2518-000/p/1FT-0004-... For servers, a comparison is even more complicated and it wouldn't be fair to just give two numbers, but I still don't think GPUs are mor…

Consumer GPUs are very cheap but prohibited to use it on datacenter.

That only applies to Nvidia GPUs.

Re: Are GPUs Worth It for ML?

#99

Post by fancyfredbot »

varunkmohan wrote: post #73
fancyfredbot wrote: post #61

Earlier quoted context omitted.

Fair point. It's not obvious from the website which model frameworks does exafunction supports, or when the last exafunction release was.

Yeah, we should have a public release very soon for people to deploy internally. We will have support for all the commonly-used frameworks and different versions.

Sounds awesome, look forward to it.

Re: Are GPUs Worth It for ML?

#100

Post by fomine3 »

atq2119 wrote: post #98
fomine3 wrote: post #77

Earlier quoted context omitted.

Consumer GPUs are very cheap but prohibited to use it on datacenter.

That only applies to Nvidia GPUs.

Ah yes but recent consumer RADEONs are not suitable for computing task (ROCm still experimental?), while Geforce is always fine for FP32 or below.
Post reply on HN