Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Alveo V70 AI inference accelerator card

xilinx.com

Post reply on HN
Print view View original

81–90 of 94 posts

Re: AMD Alveo V70 AI inference accelerator card

#81

Post by UncleEntity »

DeepYogurt wrote: post #78
kombine wrote: post #50

Earlier quoted context omitted.

4090 (or 3090, 1080Ti and so on) is a high-end consumer GPU, but at the same time it is an entry level GPU for AI researchers. Don't forget that workstation cards (RTX 8000) let alone server-grade GPUs such as A100 are an order of magnitude more expensive.

That does not sound like "entry level" research to me.

IDK, I want to train up some AI and have the choice of using google colab or buying a GPU that can do it within a reasonable timeframe.

Not going to be spending $10k like the tortoise-tts guy (was looking into that project last night) but $2k might be doable for a hobby project. Plus I’d have a computer at the end.

Re: AMD Alveo V70 AI inference accelerator card

#82

Post by dathinab »

nl wrote: post #74
dathinab wrote: post #72

Earlier quoted context omitted.

> product comparable to NVIDIA 4090 no, they need a product good at training and gpu compute at a reasonable price that product doesn't need to be good at rendering, ray tracing and similar sure students and some independent contractors probably love getting both a good graphic card and a CUDA card in one and it makes it easier for people to experiment with it but company PCs normally ban playing games on company PCs…

Long time ML worker here. People work in one of 3 ways: 1) Consumer Nvidia GPU cards on custom PCs 2) Self hosted shared server 3) Cloud infrastructure. There is no "GPU compute only card" that is widely used outside servers. > company PCs normally ban playing games on company PCs and the overlap of "needing max GPU compute" and "needing complicated 3D rendering tasks" is limited. The "don't play games thing" isn't a…

my point is a card bought for doing GPU computer not being able to work as a normal graphic card is not a problem

I'm aware that currently GPU compute only cards are not widly used outside of the server space.

But that's not because people need the consumer GPU features (bedsides video decode) but because the economics of availability and cost lead to consumer GPUs being the best option (and this economic effects don't just apply to customers but also Nvidia itself).

Re: AMD Alveo V70 AI inference accelerator card

#83

Post by scottlamb »

zamadatix wrote: post #80
scottlamb wrote: post #35

Earlier quoted context omitted.

Still depends. As it happens, I'm developing my own open source NVR software, [1] so I know a bit about this. Some cameras are fairly good about this, supporting the following features: * "Temporal SVC", in which the frame dependencies are structured so you can discard down to 1/2 or 1/4th of the nominal frame rate and still decode the remainder. * Three output streams, which you could configure for say forensics (hi…

Price wise if this card is $5,000 that's $52 per where you don't need any onboard smarts handled by the camera in a space where commercial cameras are hundreds of dollars to buy or replace to have the particular smarts you're looking for that day. I've done a few PoCs in the smart city/smart retail space they are advertising here and they pretty much end up falling into the "everything must be pre-processed as much a…

> Price wise if this card is $5,000 that's $52 per where you don't need any onboard smarts handled by the camera in a space where commercial cameras are hundreds of dollars to buy or replace to have the particular smarts you're looking for that day.

Good point. At that scale, the price might make sense. (I'd still hesitate to buy this card, though. Based on experience with Amazon VT1 instances, I don't have any faith in Xilinx's software quality.)

There are much lower-cost solutions if you don't need that many cameras, e.g.:

* The Coral TPU is nice and cheap. I keep hoping to see a new version and/or someone making M.2/PCIe cards with several of these chips on it. It doesn't do the video decoding, though, so you need other hardware for that.

* There was an Axelera card just announced. [1] I'm curious to read the reviews when it actually ships to folks.

* The newer Rockchip SoCs advertise decent video decoding and some ML acceleration. I have one and will be trying it out sooner or later.

> The onboard camera processing is usually about justifying a cloud pitch ("we use data to send video when something interesting happens" or "we send only the best picture of the face in HD to save bandwidth but still be able to ID them later") not so much letting you go in and solve your own problem.

My software's more aimed at the home/hobbyist side of things. There some folks go with the canned/cloud stuff (Ring/Nest/whatever) similar to what you're saying. Some do everything at home with e.g. BlueIris and use the on-camera ML stuff as it is. The lack of flexibility (mostly due to closed-source, low-quality software IMHO) is a real problem though. Some folks use something like Frigate that does on-NVR analytics, and I'll eventually add that feature to my own software.

> I also have a sneaking suspicion using lower channel counts let you raise the FPS but the max of 96 channels is the hard limit, tuned to allow up to use cases like recognition from unprocessed feeds but the documentation access seems to be a manual approval process so I can't verify for sure.

I bet you're right.

[1] https://www.cnx-software.com/2023/01/02/150-axelera-m2-ai-ac...

Re: AMD Alveo V70 AI inference accelerator card

#84

Post by rubatuga »

dhruvdh wrote: post #77
tgtweak wrote: post #69

Sad to see amds ROCm efforts essentially abandoned. They were close to universal interop for cudnn and cuda on amd (and other!) Architectures. Hopefully Intel takes a stab at it with their ARC line out now.

Abandoned? What?

In Their consumer card line

Re: AMD Alveo V70 AI inference accelerator card

#85

Post by vosper »

ClumsyPilot wrote: post #79
vosper wrote: post #68

Earlier quoted context omitted.

Well they definitely can, and do. They have made the decision that the reward isn't worth the risk. The idea (not aiming at you here, you didn't say this) that senior leadership at AMD is unaware of NVIDIA's lead in this space, and haven't repeatedly considered whether to invest in competing, is absurd. Likewise the idea that anyone outside of AMD understands better than AMD does what it would take in terms of invest…

> The idea that senior leadership at AMD is unaware.. Senior leadership at AMD isn't dumb. Lets try this with another company: The idea that leadership at Lehmon Brothers is unaware of the fact that they are trading subprime loans is absurd! The leadership isnt dumb The Idea that leadership at Being is unaware of safety issues with 737 Max is absurd! How could you suggest that anyone outside boesing understands bette…

The fact that a couple of other companies have had dumb leadership in no way proves that AMD's is dumb. You're essentially claiming that because Lehman and Boeing had dumb leadership, all senior leadership at all companies is dumb (because there's nothing linking AMD to these other companies except that they're all companies). And that is an absurd claim.

Re: AMD Alveo V70 AI inference accelerator card

#86

Post by gymbeaux »

lostmsu wrote: post #70
vosper wrote: post #68

Earlier quoted context omitted.

Well they definitely can, and do. They have made the decision that the reward isn't worth the risk. The idea (not aiming at you here, you didn't say this) that senior leadership at AMD is unaware of NVIDIA's lead in this space, and haven't repeatedly considered whether to invest in competing, is absurd. Likewise the idea that anyone outside of AMD understands better than AMD does what it would take in terms of invest…

They may not be dumb, but they may have narrow vision (which I suppose is still dumb). Bad decisions in top tiers of the largest world companies are not unheard of.

Yeah but no one can deny Lisa Su is an MVP and knows what she’s doing. She’s borderline overqualified for CEO of AMD IMO

Re: AMD Alveo V70 AI inference accelerator card

#87

Post by lostmsu »

gymbeaux wrote: post #86
lostmsu wrote: post #70

Earlier quoted context omitted.

They may not be dumb, but they may have narrow vision (which I suppose is still dumb). Bad decisions in top tiers of the largest world companies are not unheard of.

Yeah but no one can deny Lisa Su is an MVP and knows what she’s doing. She’s borderline overqualified for CEO of AMD IMO

> but no one can deny Lisa Su is an MVP and knows what she’s doing

In her area. We have no real info on how proficient she is in other areas like ML.

Re: AMD Alveo V70 AI inference accelerator card

#88

Post by _carbyau_ »

vosper wrote: post #68
lostmsu wrote: post #66

Earlier quoted context omitted.

> AMD has finite resources, like any company, and they’ve been focusing on CPU/datacenter dominance Why can't large companies tap the investment market? E.g. they could sell bonds to fund it, borrow, etc.

Well they definitely can, and do. They have made the decision that the reward isn't worth the risk. The idea (not aiming at you here, you didn't say this) that senior leadership at AMD is unaware of NVIDIA's lead in this space, and haven't repeatedly considered whether to invest in competing, is absurd. Likewise the idea that anyone outside of AMD understands better than AMD does what it would take in terms of invest…

Maybe they are not dumb. But my next GPU will be nVidia because of their decision. Hence I am disappointed at not having a competitive rival.

I may not know better. But I know what I like.

Re: AMD Alveo V70 AI inference accelerator card

#89

Post by Roark66 »

If this is based on fpga tech (xilinx) I don't think it will have a cost/benefit edge over asics. Why not do their own TPU like Google did? Nowadays even embedded MCUs come with AI accelerators (last I heard was 5TOPS in a banana pi-cm4 board - that is sufficient for object detection stuff and perhaps even more).

Re: AMD Alveo V70 AI inference accelerator card

#90

Post by psychphysic »

AnonMO wrote: post #67
psychphysic wrote: post #65

Did I miss something did AMD buy Xilinx? Makes sense I suppose after Intel bought Altera. Who owns lattice?

In the time it took you to write this you could have searched your two question on google clicked the first wiki link and got your answer

Bah! I'll go back to voicing all my thoughts to chagpt
Post reply on HN