Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Think About GPUs

jax-ml.github.io

Post reply on HN
Print view View original

51–60 of 127 posts

Re: How to Think About GPUs

#51

Post by bee_rider »

tormeh wrote: post #19

I find it very hard to justify investing time into learning something that's neither open source nor has multiple interchangeable vendors. Being good at using Nvidia chips sounds a lot like being an ABAP consultant or similar to me. I realize there's a lot of money to be made in the field right now, but IIUC historically this kind of thing has not been a great move.

I think I’d rather get familiar with cupy or Jax or something. Blas/lapack wrappers will never go out of style. It is a subset of the sort of stuff you can do on a GPU but it seems like a nice effort:functionality reward ratio.

Re: How to Think About GPUs

#52

Post by the__alchemist »

tormeh wrote: post #19

I find it very hard to justify investing time into learning something that's neither open source nor has multiple interchangeable vendors. Being good at using Nvidia chips sounds a lot like being an ABAP consultant or similar to me. I realize there's a lot of money to be made in the field right now, but IIUC historically this kind of thing has not been a great move.

The principles of parallel computing, and how they work at hardware and driver levels are more broad. Some parts of it are provincial (Strong province though...), and others are more general.

It's hard to find skills that don't have a degree of provincialism. It's not a great feeling, but you more on. IMO, don't over-idealize the concept of general-knowledge to your detriment.

I think we can also untangle the open-source part from the general/provincial. There is more to the world worth exploring.

Re: How to Think About GPUs

#53

Post by kjellsbells »

woooooo wrote: post #47
NikolaNovak wrote: post #45

Earlier quoted context omitted.

I mean, I'm in Toronto Canada, a fairly big city and market, and have an open seat for a couple of good senior Oracle DBAs pretty much constantly. The market may have reduced over decades but there's still more demand than supply. And the core DBA skills are transferable to other RDBMS as well. While I agree that some niche technologies are fleeting, it's perhaps not the best example :-)

That's actually interesting! My experience is different, especially compared to the late 90s and early 00s, most people avoid Oracle if they can. But yes, its always worth having someone who's job is to think about the database if it's your lynchpin.

Well, there's the difference. Maybe demand has collapsed for the kind of people who knew how to tune the Oracle SGA and get their laughable CLI client to behave, but the market for people who structurally understood the best ways to organize, insert and pull data back out is still solid.

Re Oracle and "big 90s names" specifically, there is a lot of it out there. Maybe it never shows up in the code interfaces HNers have to exercise in their day jobs, but the tech, for better or worse, is massively prevalent in the everyday world of transit systems and payroll and payment...ie all the unsexy parts of modern life.

Re: How to Think About GPUs

#55

Post by raincole »

tormeh wrote: post #19

I find it very hard to justify investing time into learning something that's neither open source nor has multiple interchangeable vendors. Being good at using Nvidia chips sounds a lot like being an ABAP consultant or similar to me. I realize there's a lot of money to be made in the field right now, but IIUC historically this kind of thing has not been a great move.

Yeah that was what I told myself a decade ago when I skipped CUDA class during college time.

Re: How to Think About GPUs

#56

Post by KeplerBoy »

gregorygoc wrote: post #9

It’s mind boggling why this resource has not been provided by NVIDIA yet. It reached the point that 3rd parties reverse engineer and summarize NV hardware to a point it becomes an actually useful mental model. What are the actual incentives at NVIDIA? If it’s all about marketing they’re doing great, but I have some doubts about engineering culture.

Nvidia has ridiculously good documentation for all of this compared to its competitors.

Re: How to Think About GPUs

#58

Post by deltaburnt »

tormeh wrote: post #19

I find it very hard to justify investing time into learning something that's neither open source nor has multiple interchangeable vendors. Being good at using Nvidia chips sounds a lot like being an ABAP consultant or similar to me. I realize there's a lot of money to be made in the field right now, but IIUC historically this kind of thing has not been a great move.

This is a JAX article, a parallel computation library that's meant to abstract away vendor specific details. Obviously if you want the most performance you need to know specifics of your hardware, but learning the high level of how a GPU vs TPU works seems like useful knowledge regardless.

Re: How to Think About GPUs

#59

Post by behnamoh »

deltaburnt wrote: post #58
tormeh wrote: post #19

I find it very hard to justify investing time into learning something that's neither open source nor has multiple interchangeable vendors. Being good at using Nvidia chips sounds a lot like being an ABAP consultant or similar to me. I realize there's a lot of money to be made in the field right now, but IIUC historically this kind of thing has not been a great move.

This is a JAX article, a parallel computation library that's meant to abstract away vendor specific details. Obviously if you want the most performance you need to know specifics of your hardware, but learning the high level of how a GPU vs TPU works seems like useful knowledge regardless.

> abstract away vendor specific details

Sounds good on paper but unfortunately I've had numerous issues with these "abstractors". For example, PyTorch had serious problems on Apple Silicon even though technically it should "just work" by hiding the implementation details.

In reality, what ends up happening is that some features in JAX, PyTorch, etc. are designed with CUDA in mind, and Apple Silicon is an afterthought.

Re: How to Think About GPUs

#60

Post by charleshn »

aschleck wrote: post #8
nickysielicki wrote: post #7

The calculation under “Quiz 2: GPU nodes“ is incorrect, to the best of my knowledge. There aren’t enough ports for each GPU and/or for each switch (less the crossbar connections) to fully realize the 450GB/s that’s theoretically possible, which is why 3.2TB/s of internode bandwidth is what’s offered on all of the major cloud providers and the reference systems. If it was 3.6TB/s, this would produce internode bottlene…

It's been a while since I thought about this but isn't the reason providers advertise only 3.2tbps because that's the limit of a single node's connection to the IB network? DGX is spec'ed to pair each H100 with a Connect-X 7 NIC and those cap out at 400gbps. 8 gpus * 400gbps / gpu = 3.2tbps. Quiz 2 is confusingly worded but is, iiuc, referring to intranode GPU connections rather than internode networking.

Yes, 450GB/s is the per GPU bandwidth in the nvlink domain. 3.2Tbps is the per-host bandwidth in the scale out IB/Ethernet domain.
Post reply on HN