Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Alveo V70 AI inference accelerator card

xilinx.com

Post reply on HN
Print view View original

61–70 of 94 posts

Re: AMD Alveo V70 AI inference accelerator card

#61

Post by marcyb5st »

hedgehog wrote: post #51
modeless wrote: post #43

Earlier quoted context omitted.

The hardware is not their major problem. They have been failing super hard at the software side of machine learning for a solid decade now. It seems like pure management incompetence to me. They need to invest a whole lot more in software, integrating their stuff directly into pytorch/TF/XLA/etc and making sure it works on consumer cards too. The investment would be paid back tenfold. The market is crying out for mor…

Last I checked they see deep learning training as a niche market, their strategy is to try to win big contracts (HPC etc) and then supply software specifically for that. Then "the community" will supply software. Having spent a bunch of time beating my head on this and related walls it's not clear to me that they're entirely wrong from an economic standpoint. Remember that 2/3 public cloud providers have their own ch…

I hope they change their minds. At least now that generative models are becoming somewhat popular. I'd love to be able to get an AMD card to run generative models, but to the best of my knowledge, they only run on Nvidia hardware

Re: AMD Alveo V70 AI inference accelerator card

#62

Post by p1esk »

gymbeaux wrote: post #57
kombine wrote: post #50

Earlier quoted context omitted.

4090 (or 3090, 1080Ti and so on) is a high-end consumer GPU, but at the same time it is an entry level GPU for AI researchers. Don't forget that workstation cards (RTX 8000) let alone server-grade GPUs such as A100 are an order of magnitude more expensive.

I was doing ML stuff on a GTX 1060 a few years ago. As with everything, it depends on what you’re doing .

Chances of you publishing something in ML improve proportionally to the amount of hardware you have access to. Or to put it another way, the less hardware you have, the smarter you have to be to publish something in ML.

Re: AMD Alveo V70 AI inference accelerator card

#63

Post by marcyb5st »

Hooray_Darakian wrote: post #41
kombine wrote: post #39

This is inference only. AMD should invest into the full AI stack starting from training. For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Honestly, I don't know why AMD aren't doing that already, they are best positioned to do that in the industry landscape.

> AMD should invest into the full AI stack starting from training. https://www.amd.com/en/graphics/servers-solutions-rocm-ml > For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Why is a high end product a requirement for entry level research?

Because of VRAM. Even a simple model for language can easily max out a 4090.

Also, ROC-M is a bit of a mess to setup. With Nvidia i just need to install cuda, cudnn and then pip install tensorflow/pytorch.

Re: AMD Alveo V70 AI inference accelerator card

#64

Post by cptskippy »

gymbeaux wrote: post #56
modeless wrote: post #43

Earlier quoted context omitted.

The hardware is not their major problem. They have been failing super hard at the software side of machine learning for a solid decade now. It seems like pure management incompetence to me. They need to invest a whole lot more in software, integrating their stuff directly into pytorch/TF/XLA/etc and making sure it works on consumer cards too. The investment would be paid back tenfold. The market is crying out for mor…

AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance, which to me is both the safer bet and the more-lucrative bet. It wasn’t that long ago that AMD was on the brink of bankruptcy (~2016), so I appreciate that they’re not trying to divide their attention. Their attempts at entering the ML space so far have been failures, and they are wise to hold off on really competing wi…

> AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance, which to me is both the safer bet and the more-lucrative bet.

Cloud/datacenter based ML is a huge growth market. Having the same software work on a consumer GPUs and enterprise ML cards is one of Nvidia's competitive advantages.

Re: AMD Alveo V70 AI inference accelerator card

#66

Post by lostmsu »

gymbeaux wrote: post #56
modeless wrote: post #43

Earlier quoted context omitted.

The hardware is not their major problem. They have been failing super hard at the software side of machine learning for a solid decade now. It seems like pure management incompetence to me. They need to invest a whole lot more in software, integrating their stuff directly into pytorch/TF/XLA/etc and making sure it works on consumer cards too. The investment would be paid back tenfold. The market is crying out for mor…

AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance, which to me is both the safer bet and the more-lucrative bet. It wasn’t that long ago that AMD was on the brink of bankruptcy (~2016), so I appreciate that they’re not trying to divide their attention. Their attempts at entering the ML space so far have been failures, and they are wise to hold off on really competing wi…

> AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance

Why can't large companies tap the investment market? E.g. they could sell bonds to fund it, borrow, etc.

Re: AMD Alveo V70 AI inference accelerator card

#68

Post by vosper »

lostmsu wrote: post #66
gymbeaux wrote: post #56

Earlier quoted context omitted.

AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance, which to me is both the safer bet and the more-lucrative bet. It wasn’t that long ago that AMD was on the brink of bankruptcy (~2016), so I appreciate that they’re not trying to divide their attention. Their attempts at entering the ML space so far have been failures, and they are wise to hold off on really competing wi…

> AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance Why can't large companies tap the investment market? E.g. they could sell bonds to fund it, borrow, etc.

Well they definitely can, and do. They have made the decision that the reward isn't worth the risk.

The idea (not aiming at you here, you didn't say this) that senior leadership at AMD is unaware of NVIDIA's lead in this space, and haven't repeatedly considered whether to invest in competing, is absurd. Likewise the idea that anyone outside of AMD understands better than AMD does what it would take in terms of investment _and opportunity cost_, is also absurd.

Senior leadership at AMD isn't dumb. The fact that they're not doing something we want doesn't make them dumb, either. Again, not aiming at you with this little rant :)

Re: AMD Alveo V70 AI inference accelerator card

#70

Post by lostmsu »

vosper wrote: post #68
lostmsu wrote: post #66

Earlier quoted context omitted.

> AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance Why can't large companies tap the investment market? E.g. they could sell bonds to fund it, borrow, etc.

Well they definitely can, and do. They have made the decision that the reward isn't worth the risk. The idea (not aiming at you here, you didn't say this) that senior leadership at AMD is unaware of NVIDIA's lead in this space, and haven't repeatedly considered whether to invest in competing, is absurd. Likewise the idea that anyone outside of AMD understands better than AMD does what it would take in terms of invest…

They may not be dumb, but they may have narrow vision (which I suppose is still dumb). Bad decisions in top tiers of the largest world companies are not unheard of.
Post reply on HN