Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Radeon VII: High-End 7nm Vega Video Card

anandtech.com

Post reply on HN
Print view View original

51–60 of 91 posts

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#51

Post by dogma1138 »

atq2119 wrote: post #43
dogma1138 wrote: post #25

Earlier quoted context omitted.

NAVI is still GCN likely the same 15-25% uptick over Polaris as we’ve seen with this new Vega, 2020-2021 is when the next gen uArch is going to come. GCN’s vector machine doesn’t scale as well as NVIDIA’s scalar architecture NVIDIA can add cores without caring about concurrency or ILP, since each core is an individually addressable scalar ALU, this is why “async” compute doesn’t benefit them as much as it does GCN ca…

Your comment is extremely confused. Let's unpack some of the confusion... > NVIDIA’s scalar architecture NVidia doesn't have a scalar architecture in any meaningful sense of the word. Individual work items execute in warps, meaning (up to) 32 items execute the same instruction, at the same PC. If you have divergence, i.e. fewer than 32 items at the same instruction, you lose ALU throughput. This is the same as in AMD…

>NVidia doesn't have a scalar architecture in any meaningful sense of the word.

CUDA threads are scalar, each CUDA core is a single scalar ALU, which CUDA cores are assigned to a warp is flexible.

>Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards.

Both AMD and NVIDIA GPUs have to do context switching. The difference is in scheduling GCN has a dedicate scheduler with 8 queues only for compute that is the ACE it sits utterly idle while doing graphics the graphics scheduler sits within each CU.

>Nowhere in GCN are there any 4-wide SIMD arrays.

Each CU contains an array of 4 SIMD units which each take a vec4 input, GCN is much closer to VILW4 than you think. Each CU also contains a scalar unit it cannot excute them in parallel.

If the SIMD units cannot be fed they sit idle, idle CUDA cores can be assigned to a different warp.

>Also nonsense. Both AMD and Nvidia have the majority of die space allocated to the compute cores

ACE takes up about 15% of the die space that is big in my book.

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#52

Post by zozbot123 »

doctorpangloss wrote: post #10
dragontamer wrote: post #2

Its an interesting design for sure. 4x Stacks of HBM2 means 1TBps memory bandwidth at 16GB. Only 60-compute units enabled (maybe 4-CUs are expected to break during manufacturing? Its a weird number for sure...). Since it shares dies with the MI50, the Radeon VII will have 1/2 speed double-precision, making this the cheapest high-performance double-precision card in existance. ------ FP16 compute is supported at doubl…

> Deep learning enthusiasts probably will stick to NVIDIA cards We've been building the wrong hardware for ML for a while now. A TPU doesn't delivery supremacy for problems over GPU hardware. It's a Google senior engineering retention and PR project. Exceedingly few problems resemble image recognition. You wouldn't be able to tell from the research, because due to the tools it's sort of the only affordable thing to d…

Matrix multiplication is the bread-and-butter of ML (both training and inference), for fairly obvious and solid reasons. Luckily for us, it's also very important in all sorts of domains, including graphics, rendering and gaming. That's all there is to it. There's no real "subsidy from the gaming industry", only a cross-subsidy situation in which common problems that are shared across domains can take advantage of the increased scale that comes from sharing the GPU-compute platform.

I'm not sure what you mean by "ML features", but AMD/ATI's dismissive attitude towards the ML and GPU-compute communities is looking quite silly by now.

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#53

Post by dragontamer »

dogma1138 wrote: post #51
atq2119 wrote: post #43

Earlier quoted context omitted.

Your comment is extremely confused. Let's unpack some of the confusion... > NVIDIA’s scalar architecture NVidia doesn't have a scalar architecture in any meaningful sense of the word. Individual work items execute in warps, meaning (up to) 32 items execute the same instruction, at the same PC. If you have divergence, i.e. fewer than 32 items at the same instruction, you lose ALU throughput. This is the same as in AMD…

>NVidia doesn't have a scalar architecture in any meaningful sense of the word. CUDA threads are scalar, each CUDA core is a single scalar ALU, which CUDA cores are assigned to a warp is flexible. >Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards. Both AMD and NVIDIA GPUs have to do co…

Unfortunately, I agree with the other poster. There are a number of inaccuracies in your post.

> Each CU contains an array of 4 SIMD units which each take a vec4 input

AMD SIMDs are (poorly named) scalar units, which are programmed using scalar arithmetic. FP16 is the only exception, as FP16 is a true SIMD operation (taking place inside a AMD scalar SIMD 2-at-a-time).

Yeah, AMD really needed to name their stuff better. "vGPRs" ("vector General Purpose Registers") are in fact scalar units, as documented in the Vega Instruction Set.

See chapter 6 for proof: https://developer.amd.com/wp-content/resources/Vega_Shader_I...

The SIMDs replaced historical SIMD-units from the 6000-series roughly 10 years ago. I bet that the name is historical in nature. But AMD's cards are scalarly programmed, just like NVidia's. There is no need to use vec4 on modern (aka: anything in the last 10 years) cards from AMD. Half2 is the only thing that gets a benefit.

> Each CU also contains a scalar unit it cannot excute them in parallel.

The "Scalar" unit in AMD GPUs is... closer to a boolean-vector unit. AMD really chose bad names for these things... But anyway, it is a 64-bit unit, where each bit is used to calculate the execution masks to the vector units.

The "Scalar" unit runs once every clock-tick. "Vector" units repeat themselves over 4-clock ticks (and therefore execute 4x slower than the Scalar unit). Ultimately, 1-Scalar unit can perform roughly the same amount of work as the 4-vector units.

See page 31 for a great example on how the sALU of Vega is used to create loops and handle divergent cases. The "sALU" is effectively the unit to calculate "if" statements and "loops".

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#54

Post by Joky »

atq2119 wrote: post #43
dogma1138 wrote: post #25

Earlier quoted context omitted.

NAVI is still GCN likely the same 15-25% uptick over Polaris as we’ve seen with this new Vega, 2020-2021 is when the next gen uArch is going to come. GCN’s vector machine doesn’t scale as well as NVIDIA’s scalar architecture NVIDIA can add cores without caring about concurrency or ILP, since each core is an individually addressable scalar ALU, this is why “async” compute doesn’t benefit them as much as it does GCN ca…

Your comment is extremely confused. Let's unpack some of the confusion... > NVIDIA’s scalar architecture NVidia doesn't have a scalar architecture in any meaningful sense of the word. Individual work items execute in warps, meaning (up to) 32 items execute the same instruction, at the same PC. If you have divergence, i.e. fewer than 32 items at the same instruction, you lose ALU throughput. This is the same as in AMD…

> NVidia doesn't have a scalar architecture in any meaningful sense of the word.

While I agree with your post in general, I'm not totally in agreement here. If you look at the context in which G80 and CUDA were introduced, it has a significant scalar dimension: for a single "thread" writing an addition between two float3 ended up being executed as 3 separate additions on the scalar components of the vector. This was, if I remember correctly, a departure from previous architecture optimized for graphic processing. In this sense the ISA is/was mostly scalar.

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#55

Post by frankchn »

vl wrote: post #50
twtw wrote: post #39

Earlier quoted context omitted.

So that's why Xeon Phi has been so successful! /s Real talk, though. Why do you think larabee is the "right" hardware?

The truth is that for many types/sizes of models AVS2 or AVS512 in Xeons is as fast and GPUs.

I am interested in seeing benchmarks regarding this. I believe this is true for models with truly huge embeddings, but otherwise if your models are even a little compute dense then GPUs are faster.

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#56

Post by brokenmachine »

jMyles wrote: post #21

Perhaps a good candidate for mining coins like LivePeer (one of 8-10 actual solid blockchain projects IMO).

>LivePeer (one of 8-10 actual solid blockchain projects IMO)

Interesting.

Can anyone suggest a good curated list of solid blockchain projects?

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#58

Post by Cybiote »

frankchn wrote: post #55
vl wrote: post #50

Earlier quoted context omitted.

The truth is that for many types/sizes of models AVS2 or AVS512 in Xeons is as fast and GPUs.

I am interested in seeing benchmarks regarding this. I believe this is true for models with truly huge embeddings, but otherwise if your models are even a little compute dense then GPUs are faster.

My experience is that you need the additional caveat of a streaming fairly homogeneous and highly parallelizable approach or the memory transfer, branching and communication overheads will eat away nearly all the GPU gains.

I've also noticed that a lot of the time, people are comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations. For a large swathe of problem types, using the fastest math library will reduce the CPU/GPU gap to less than an order of magnitude or less.

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#59

Post by atq2119 »

dragontamer wrote: post #53
dogma1138 wrote: post #51

Earlier quoted context omitted.

>NVidia doesn't have a scalar architecture in any meaningful sense of the word. CUDA threads are scalar, each CUDA core is a single scalar ALU, which CUDA cores are assigned to a warp is flexible. >Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards. Both AMD and NVIDIA GPUs have to do co…

Unfortunately, I agree with the other poster. There are a number of inaccuracies in your post. > Each CU contains an array of 4 SIMD units which each take a vec4 input AMD SIMDs are (poorly named) scalar units, which are programmed using scalar arithmetic. FP16 is the only exception, as FP16 is a true SIMD operation (taking place inside a AMD scalar SIMD 2-at-a-time). Yeah, AMD really needed to name their stuff bette…

> The "Scalar" unit in AMD GPUs is... closer to a boolean-vector unit. AMD really chose bad names for these things... But anyway, it is a 64-bit unit, where each bit is used to calculate the execution masks to the vector units.

This is a really cool way to put it, though I'd point out that the scalar unit is actually also used for (uniform) arithmetic, especially on pointers / indices. For example, if you have memory accesses of the form uniform_base + stride * thread_id, you can use the scalar ALU to compute / manipulate the uniform_base part.

If you think of the GCN compute units as CPU cores with a really wide SIMD + powerful scatter/gather + texture sampling unit, the chosen terminology of scalar/vector ALUs makes sense.

The difficulty is in mapping the (work item-centric) programming model that developers see to the hardware, but that's conceptually fairly similar to what ISPC does.

Re: AMD Radeon VII: High-End 7nm Vega Video Card

#60

Post by atq2119 »

dogma1138 wrote: post #51
atq2119 wrote: post #43

Earlier quoted context omitted.

Your comment is extremely confused. Let's unpack some of the confusion... > NVIDIA’s scalar architecture NVidia doesn't have a scalar architecture in any meaningful sense of the word. Individual work items execute in warps, meaning (up to) 32 items execute the same instruction, at the same PC. If you have divergence, i.e. fewer than 32 items at the same instruction, you lose ALU throughput. This is the same as in AMD…

>NVidia doesn't have a scalar architecture in any meaningful sense of the word. CUDA threads are scalar, each CUDA core is a single scalar ALU, which CUDA cores are assigned to a warp is flexible. >Nvidia literally cannot do async compute in the same way as AMD, because their micro-architecture cannot switch between graphics and compute on as finely grained a basis as GCN cards. Both AMD and NVIDIA GPUs have to do co…

> idle CUDA cores can be assigned to a different warp.

I have a feeling you're confusing lanes vs. cores here. Individual "threads" (work items) cannot migrate between warps, and every "core" only issues one instruction at a time (per execution port, in some of Nvidia's architectures). Each execution port is a SIMD execution port executing that instruction for multiple work items. If the work items of a warp diverge, some of the lanes of those SIMD execution units will be idle.

This is the exact same behavior as on AMD's GCN, though obviously the terminology is different and some of the other details may be wired differently.

> ACE takes up about 15% of the die space that is big in my book.

I have a feeling that you're just making up numbers here. Can you point to evidence?

Post reply on HN