Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Alveo V70 AI inference accelerator card

xilinx.com

Post reply on HN
Print view View original

71–80 of 94 posts

Re: AMD Alveo V70 AI inference accelerator card

#71

Post by modeless »

vosper wrote: post #68
lostmsu wrote: post #66

Earlier quoted context omitted.

> AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance Why can't large companies tap the investment market? E.g. they could sell bonds to fund it, borrow, etc.

Well they definitely can, and do. They have made the decision that the reward isn't worth the risk. The idea (not aiming at you here, you didn't say this) that senior leadership at AMD is unaware of NVIDIA's lead in this space, and haven't repeatedly considered whether to invest in competing, is absurd. Likewise the idea that anyone outside of AMD understands better than AMD does what it would take in terms of invest…

The idea that AMD management can't possibly have made any bad decisions is the absurd thing here. It's entirely possible that AMD carefully considered Nvidia's position, carefully considered their strategy, and confidently made the wrong decision. It happens all the time in all sorts of companies.

I think it's very clear with the benefit of hindsight that not investing enough into the software side of deep learning early on was a bad decision. But it was obvious to me even at the time and I said as much to anyone who would listen (e.g. seven years ago https://news.ycombinator.com/item?id=12258027)

Re: AMD Alveo V70 AI inference accelerator card

#72

Post by dathinab »

kombine wrote: post #39

This is inference only. AMD should invest into the full AI stack starting from training. For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Honestly, I don't know why AMD aren't doing that already, they are best positioned to do that in the industry landscape.

> product comparable to NVIDIA 4090

no, they need a product good at training and gpu compute at a reasonable price

that product doesn't need to be good at rendering, ray tracing and similar

sure students and some independent contractors probably love getting both a good graphic card and a CUDA card in one and it makes it easier for people to experiment with it but company PCs normally ban playing games on company PCs and the overlap of "needing max GPU compute" and "needing complicated 3D rendering tasks" is limited.

through having 1 product instead of two does make supply chain and pricing easier

but then 4090 is by now in a price range where students are unlikely to afford it and people will think twice about buying it just to play around with GPU compute.

So e.g. the 7900XTX having somewhat comparable GPU compute usability then a 4080 would have been good enough for the non company use case, where a dedicated compute-per-money cheaper GPU compute only card would be preferable for the company use case I think.

Re: AMD Alveo V70 AI inference accelerator card

#73

Post by nl »

marcyb5st wrote: post #61
hedgehog wrote: post #51

Earlier quoted context omitted.

Last I checked they see deep learning training as a niche market, their strategy is to try to win big contracts (HPC etc) and then supply software specifically for that. Then "the community" will supply software. Having spent a bunch of time beating my head on this and related walls it's not clear to me that they're entirely wrong from an economic standpoint. Remember that 2/3 public cloud providers have their own ch…

I hope they change their minds. At least now that generative models are becoming somewhat popular. I'd love to be able to get an AMD card to run generative models, but to the best of my knowledge, they only run on Nvidia hardware

No personal experience, but you can actually get Stable Diffusion to run on AMD cards.

It uses DirectML on Windows: https://gist.github.com/averad/256c507baa3dcc9464203dc14610d... This is thanks to Microsoft, not AMD.

On Linux you can use ROCm: https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GP...

The horrible install processes and what a mess this is is all down to AMD.

Re: AMD Alveo V70 AI inference accelerator card

#74

Post by nl »

dathinab wrote: post #72
kombine wrote: post #39

This is inference only. AMD should invest into the full AI stack starting from training. For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Honestly, I don't know why AMD aren't doing that already, they are best positioned to do that in the industry landscape.

> product comparable to NVIDIA 4090 no, they need a product good at training and gpu compute at a reasonable price that product doesn't need to be good at rendering, ray tracing and similar sure students and some independent contractors probably love getting both a good graphic card and a CUDA card in one and it makes it easier for people to experiment with it but company PCs normally ban playing games on company PCs…

Long time ML worker here. People work in one of 3 ways:

1) Consumer Nvidia GPU cards on custom PCs

2) Self hosted shared server

3) Cloud infrastructure.

There is no "GPU compute only card" that is widely used outside servers.

> company PCs normally ban playing games on company PCs and the overlap of "needing max GPU compute" and "needing complicated 3D rendering tasks" is limited.

The "don't play games thing" isn't a factor. Most companies just buy a 4090 or whatever, and if they have to tell staff not to play games, they say "don't play games". Fortnight runs just fine on pretty much anything anyway.

Re: AMD Alveo V70 AI inference accelerator card

#75

Post by hedgehog »

marcyb5st wrote: post #61
hedgehog wrote: post #51

Earlier quoted context omitted.

Last I checked they see deep learning training as a niche market, their strategy is to try to win big contracts (HPC etc) and then supply software specifically for that. Then "the community" will supply software. Having spent a bunch of time beating my head on this and related walls it's not clear to me that they're entirely wrong from an economic standpoint. Remember that 2/3 public cloud providers have their own ch…

I hope they change their minds. At least now that generative models are becoming somewhat popular. I'd love to be able to get an AMD card to run generative models, but to the best of my knowledge, they only run on Nvidia hardware

I wouldn't hold my breath, and anyway at this point NVIDIA has faster chips and more supported software all the way down the stack. My previous startup tried to solve some of these problems and we built what is as far as I know still the only reasonably complete device-portable deep learning framework. Today something like an RTX 3070 is a good budget option for small experiments and you can always lean on a cloud provider if you need more compute temporarily. Hard to beat a TPU pod when you're in a hurry.

Re: AMD Alveo V70 AI inference accelerator card

#76

Post by hedgehog »

nl wrote: post #73
marcyb5st wrote: post #61

Earlier quoted context omitted.

I hope they change their minds. At least now that generative models are becoming somewhat popular. I'd love to be able to get an AMD card to run generative models, but to the best of my knowledge, they only run on Nvidia hardware

No personal experience, but you can actually get Stable Diffusion to run on AMD cards. It uses DirectML on Windows: https://gist.github.com/averad/256c507baa3dcc9464203dc14610d... This is thanks to Microsoft, not AMD. On Linux you can use ROCm: https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GP... The horrible install processes and what a mess this is is all down to AMD.

I don't have any experience with DirectML but it sounds promising.

Re: AMD Alveo V70 AI inference accelerator card

#78

Post by DeepYogurt »

kombine wrote: post #50
Hooray_Darakian wrote: post #41

Earlier quoted context omitted.

> AMD should invest into the full AI stack starting from training. https://www.amd.com/en/graphics/servers-solutions-rocm-ml > For this they need a product comparable to NVIDIA 4090, so that entry level researchers could use their hardware. Why is a high end product a requirement for entry level research?

4090 (or 3090, 1080Ti and so on) is a high-end consumer GPU, but at the same time it is an entry level GPU for AI researchers. Don't forget that workstation cards (RTX 8000) let alone server-grade GPUs such as A100 are an order of magnitude more expensive.

That does not sound like "entry level" research to me.

Re: AMD Alveo V70 AI inference accelerator card

#79

Post by ClumsyPilot »

vosper wrote: post #68
lostmsu wrote: post #66

Earlier quoted context omitted.

> AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance Why can't large companies tap the investment market? E.g. they could sell bonds to fund it, borrow, etc.

Well they definitely can, and do. They have made the decision that the reward isn't worth the risk. The idea (not aiming at you here, you didn't say this) that senior leadership at AMD is unaware of NVIDIA's lead in this space, and haven't repeatedly considered whether to invest in competing, is absurd. Likewise the idea that anyone outside of AMD understands better than AMD does what it would take in terms of invest…

> The idea that senior leadership at AMD is unaware.. Senior leadership at AMD isn't dumb.

Lets try this with another company:

The idea that leadership at Lehmon Brothers is unaware of the fact that they are trading subprime loans is absurd! The leadership isnt dumb

The Idea that leadership at Being is unaware of safety issues with 737 Max is absurd! How could you suggest that anyone outside boesing understands better than they do the risks involved?

Re: AMD Alveo V70 AI inference accelerator card

#80

Post by zamadatix »

scottlamb wrote: post #35
zamadatix wrote: post #31

Earlier quoted context omitted.

If your source is 96 YouTube videos sure, if it's 96 CCTV cameras it's different.

Still depends. As it happens, I'm developing my own open source NVR software, [1] so I know a bit about this. Some cameras are fairly good about this, supporting the following features: * "Temporal SVC", in which the frame dependencies are structured so you can discard down to 1/2 or 1/4th of the nominal frame rate and still decode the remainder. * Three output streams, which you could configure for say forensics (hi…

Price wise if this card is $5,000 that's $52 per where you don't need any onboard smarts handled by the camera in a space where commercial cameras are hundreds of dollars to buy or replace to have the particular smarts you're looking for that day. I've done a few PoCs in the smart city/smart retail space they are advertising here and they pretty much end up falling into the "everything must be pre-processed as much as possible and sent to the cloud" or "everything must be dumb and sent to the central recorder" buckets as anything in the middle creates a bad cost balance where you're neither optimising hardware+simplicity costs or data+cloud costs. I'll admit though I don't normally go out to sell cameras all day it's just something we've added as clients in part of a larger connectivity rework (CBRS/LTE/Wi-Fi/GPON/traditional wired) and we typically partner up with some specialized company on the video processing use case. The onboard camera processing is usually about justifying a cloud pitch ("we use data to send video when something interesting happens" or "we send only the best picture of the face in HD to save bandwidth but still be able to ID them later") not so much letting you go in and solve your own problem. One exception I ran into was license plates at a car wash outfit where they were able to send the plate numbers back to their main app but that probably came from being a pre-baked solution for road tolls.

I also have a sneaking suspicion using lower channel counts let you raise the FPS but the max of 96 channels is the hard limit, tuned to allow up to use cases like recognition from unprocessed feeds but the documentation access seems to be a manual approval process so I can't verify for sure.

Post reply on HN