Making AMD GPUs competitive for LLM inference (2023)

While defects ordinarily reduce yields, Intel put plenty of redundant transistors into the silicon. This is ordinarily not possible to estimate, but Tom Petersen reported in his interview with hardware unboxed that they did not count those when reporting the transistor count. Given that the density based on reported transistors is about 40% less than the density others get from the same process and the silicon in GPUs is already fairly redundant, they likely have a backup component for just about everything on the die. The consequence is that they should be able to use at least 99% of those dies even after tossing unusable dies, such that the $57 per die figure is likely correct.

As for the rest of the card, there is not much in it that would not be part of the price of an $80 Asrock motherboard. The main thing would be the bundled game, which they likely can get in bulk at around $5 per copy. This seems reasonable given how much Epic games pays for their giveaways:

https://x.com/simoncarless/status/1389297530341519362

That brings the total cost to $190. If we assume Asrock and the retailer both have a 10% margin on the $80 motherboard used as a substitute for the costs of the rest of the things, then it is $174. Then we need to add margins for board partners and the retailers. If we assume they both get 10% of the $250, then that leaves a $26 profit for Intel, provided that they have economics of scale such that the $80 motherboard approximation for the rest of the cost of the graphics card is accurate.

That is about a 10% margin for Intel. That is not a huge margin, but provided enough sales volume (to match the sales volume Asrock gets on their $80 motherboards), Intel should turn a profit on these versus not selling these at all. Interestingly, their board partners are not able/willing to hit the $250 MSRP and the closest they come to it is $260 so Intel is likely not sharing very much with them.

It should be noted that Tom Petersen claimed during his hardware unboxed interview that they were not making money on these. However, that predated the B580 being a hit and likely relied on expected low production volumes due to low sales projections. Since the B580 is a hit and napkin math says it is profitable as long as they build enough of them, I imagine that they are ramping production to meet demand and reach profitability.

At 10m3s in the following video, he claims to add a 60% margin by multiplying by 1.6, but in reality is adding a 37.5 margin and needed to multiply by 2.5 to add a 60% margin. This can be calculated by calculating Cost Scaling Factor = 1 / (1 - Normalized Profit Margin):

2.5 = 1 / (1 - 0.6)

1.6 = 1 / (1 - 0.375)

https://youtu.be/pq5G4mPOOPQ

At 48m13s in the following video, he claims that Intel’s B580 is 80% worse than Nvidia’s hardware. He took the 4070 Ti as being 82% better than the 2080 SUPER, assumed based on leaks from his reviewer friends that the B580 was about at the performance of the 2080 SUPER and then claimed that the B580 would be around 80% worse than the 4070 Ti. Unfortunately for him, that is 45% worse, not 80% worse. His chart is from Techpowerup and if he had taken the time to do some math (1 - 1/(1 + 0.82) ~ 0.45), or clicked to the 2080 SUPER page, he would have seen it has 55% of the performance of the 4070 Ti, which is 45% worse:

https://youtu.be/-lv52n078dw

At 1m2s in the following video, he makes a similar math mistake by saying that the B580 has 8% better price/performance than the RTX 3060 when in fact it is 9% better. He mistakenly equated the RTX 3060 being 8% worse than the B580 to mean that it is 8% better, but math does not work that way. Luckily for him, the math error is small here, but he still failed to do math correctly and his reasoning grows increasingly faulty with the scale of his math errors. What he should have done that gives the correct normalized factor is:

1.09 ~ 1 / (1 - 0.08)

A factor of 1.09 better is 9% better.

https://youtu.be/3jy6GDGzgbg

He not just fails at mathematical reasoning, but lacks a basic understanding of how hardware manufacturing works. He said that if Intel loses $20 per card in low production volumes, then making 10 million cards will result in a $200 million loss. In reality, things become cheaper due to economics of scale and simple napkin math shows that they can turn a profit on these cards:

https://news.ycombinator.com/item?id=42505496

His $20 loss per card remark is at 11m40s:

https://youtu.be/3jy6GDGzgbg

His behavior is consistent with being on a vendetta rather than being a technology journalist. For example, at 55m13s in the following video, he puts words in Tom Petersen’s mouth and then with a malicious smile on his mouth, cheers while claiming that Tom Petersen declared discrete ARC cards to be dead when Tom Petersen said nothing of the kind. Earlier in the same video at around 44m14s, he calls Tom Petersen a professional liar. However, he sees no problem expecting people to believe words he shoved into the “liar’s” mouth:

https://youtu.be/xVKcmGKQyXU

If you scrutinize his replies to criticism in his comments section, you would see he is dodging criticism of the actual issues with his coverage while saying “I was right about ” or “facts don’t care about your feelings”. You would also notice that he is copy and pasting the same statements rather than writing replies addressing the details of the complaints. To be clear, I am paraphrasing in those two quotes.

He also shows contempt for his viewers that object to his behavior in the following video around 18m53s where he calls them “corporate cheerleaders”:

https://youtu.be/pq5G4mPOOPQ

In short, Tom at MLID is unable to do mathematical reasoning, does not understand how hardware manufacturing works, has a clear vendetta against Intel’s discrete graphics, is unable to take constructive criticism and lashes out at those who try to tell him when he is wrong. I suggest being skeptical of anything he says about Intel’s graphics division.

The Internet is a machine that highly simplifies the otherwise complex technical challenge of wide-casting ignorance. It wide-casts wisdom too, but it's an exercise for the reader to distinguish them.