Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jeff Dean responds to EDA industry about AlphaChip

twitter.com

Post reply on HN
Print view View original

81–90 of 221 posts

Re: Jeff Dean responds to EDA industry about AlphaChip

#81

Post by alexey-salmin »

bsder wrote: post #68
throwaway2037 wrote: post #63

Earlier quoted context omitted.

> EDA companies are garbage I don't understand this comment. Can you please explain? Are they unethical? Or do they write poor software?

Yes and yes. EDA companies are gatekeeping monopolies. They absolutely abuse their monopoly position to extract huge chunks of money out of companies, and are pretty much single-handedly responsible for the fact that the hardware startup ecosystem is moribund compared to that of the software startup ecosystem. They have been horrible liars about performance and benchmarketing for decades. They dragged their feet mise…

> pretty much single-handedly responsible for the fact that the hardware startup ecosystem is moribund compared to that of the software startup ecosystem.

This was the case before EDA companies even appeared. Hardware is hard because it's manufacturing. You can't "iterate quickly", every iteration costs millions of dollars and so does every mistake.

Re: Jeff Dean responds to EDA industry about AlphaChip

#82

Post by BoingBoomTschak »

make3 wrote: post #77
vighneshiyer wrote: post #75

Earlier quoted context omitted.

> One key argument in the rebuttal against the ISPD article is that the resources used in their comparison were significantly smaller. To me, this point alone seems sufficient to question the validity of the ISPD work's conclusions. What are your thoughts on this? I believe this is a fair criticism, and it could be a reason why the ISPD Tensorboard shows divergence during training for some RTL designs. The ISPD autho…

[flagged]

> EQ

Using a fantasy concept invented by a science journalist doesn't help your posts, you know. Protip: it's just empathy + regular intelligence.

Re: Jeff Dean responds to EDA industry about AlphaChip

#83

Post by nemonemo »

vighneshiyer wrote: post #75
nemonemo wrote: post #70

Earlier quoted context omitted.

In the conclusion of the article, you said: "While I concede that there are things the ISPD authors could have done better, their conclusion is still sound. The Nature authors do not address the fact that CMP and AutoDMP outperform CT with far less runtime and compute requirements." One key argument in the rebuttal against the ISPD article is that the resources used in their comparison were significantly smaller. To…

> One key argument in the rebuttal against the ISPD article is that the resources used in their comparison were significantly smaller. To me, this point alone seems sufficient to question the validity of the ISPD work's conclusions. What are your thoughts on this? I believe this is a fair criticism, and it could be a reason why the ISPD Tensorboard shows divergence during training for some RTL designs. The ISPD autho…

Thank you for your thoughtful response. Acknowledging potential biases openly in a public forum is never easy, and in my view, it adds credibility to your words compared to leaving such matters as implicit insinuations.

That said, on page 8, the paper says that 'standard licensing agreements with commercial vendors prohibit public comparison with their offerings.' Given this inherent limitation, what alternative approach could have been taken to enable a more meaningful comparison between CT and CMP?

Re: Jeff Dean responds to EDA industry about AlphaChip

#84

Post by bsder »

alexey-salmin wrote: post #81
bsder wrote: post #68

Earlier quoted context omitted.

Yes and yes. EDA companies are gatekeeping monopolies. They absolutely abuse their monopoly position to extract huge chunks of money out of companies, and are pretty much single-handedly responsible for the fact that the hardware startup ecosystem is moribund compared to that of the software startup ecosystem. They have been horrible liars about performance and benchmarketing for decades. They dragged their feet mise…

> pretty much single-handedly responsible for the fact that the hardware startup ecosystem is moribund compared to that of the software startup ecosystem. This was the case before EDA companies even appeared. Hardware is hard because it's manufacturing. You can't "iterate quickly", every iteration costs millions of dollars and so does every mistake.

> Hardware is hard because it's manufacturing. You can't "iterate quickly", every iteration costs millions of dollars and so does every mistake.

This is true for injection molding and yet we do that all the time in small businesses.

A mask set for an older technology can be in the range of $50K-$100K. That's right about the same price as injection molds.

The main difference is that Solidworks is about $25K while Cadence, et al, is about a megabuck.

Re: Jeff Dean responds to EDA industry about AlphaChip

#86

Post by oesa »

In the tweet Jeff Dean says that Cheng at al. failed to follow the steps required to replicate the work of the Google researchers.

Specifically:

> In particular the authors did no pre-training (despite pre-training being mentioned 37 times in our Nature article), robbing our learning-based method of its ability to learn from other chip designs

But in the Circuit Training Google repo[1] they specifically say:

> Our results training from scratch are comparable or better than the reported results in the paper (on page 22) which used fine-tuning from a pre-trained model.

I may be misunderstanding something here, but which one is it? Did they mess up when they did not pre-train or they followed the "steps" described in the original repo and tried to get a fair reproduction?

Also, the UCSD group had to reverse-engineer several steps to reproduce the results so it seems like the paper's results weren't reproducible by themselves.

[1]: https://github.com/google-research/circuit_training/blob/mai...

Re: Jeff Dean responds to EDA industry about AlphaChip

#87

Post by vighneshiyer »

nemonemo wrote: post #83
vighneshiyer wrote: post #75

Earlier quoted context omitted.

> One key argument in the rebuttal against the ISPD article is that the resources used in their comparison were significantly smaller. To me, this point alone seems sufficient to question the validity of the ISPD work's conclusions. What are your thoughts on this? I believe this is a fair criticism, and it could be a reason why the ISPD Tensorboard shows divergence during training for some RTL designs. The ISPD autho…

Thank you for your thoughtful response. Acknowledging potential biases openly in a public forum is never easy, and in my view, it adds credibility to your words compared to leaving such matters as implicit insinuations. That said, on page 8, the paper says that 'standard licensing agreements with commercial vendors prohibit public comparison with their offerings.' Given this inherent limitation, what alternative appr…

So I'm not sure what Google is referring to here. As you can see in the ISPD paper (https://vlsicad.ucsd.edu/Publications/Conferences/396/c396.p...) on page 5, they openly compare Cadence CMP with AutoDMP and other algorithims quantitatively. The only obfuscation is with the proprietary GF12 technology, where they can't provide absolute numbers, but only relative ones. Comparison against commercial tools is actually a common practice in academic EDA CAD papers, although usually the exact tool vendor is obfuscated. CAD tool vendors have actually gotten more permissive about sharing tool data and scripts in public over the past few years. However, PDKs have always been under NDAs and are still very restrictive.

Perhaps the Cadence license agreement signed by a corporation is different than the one signed by a university. In such a case, they could partner with a university. But I doubt their license agreement prevents any public comparison. For example, see the AutoDMP paper from NVIDIA (https://d1qx31qr3h6wln.cloudfront.net/publications/AutoDMP.p...) where on page 7 they openly benchmark their tool against Cadence Innovus. My suspicion is they wish to keep details about the TPU blocks they evaluated under tight wraps.

Re: Jeff Dean responds to EDA industry about AlphaChip

#88

Post by gabegobblegoldi »

throwaway2037 wrote: post #64
stackghost wrote: post #48

Earlier quoted context omitted.

>peer reviewed research published in journals Peer review doesn't mean as much as Elsevier would like you to believe. Plenty of peer-reviewed research is absolute trash.

All of the highest impact papers authored by DeepMind and Google Brain have appeared in Nature, which is the gold standard for peer-reviewed natural science research. What exactly are you trying to claim about Google's peer-reviewed papers?

Peer review is not designed to combat fraud.

Re: Jeff Dean responds to EDA industry about AlphaChip

#89

Post by gabegobblegoldi »

throwaway2037 wrote: post #66
bsder wrote: post #36

Earlier quoted context omitted.

There are benchmarks in this space. You can also bring your chip designs into the open and show what happens with different tools. You can run the algorithm on the placed designs that you sponsor for open source VLSI to show how much better they are. None of this has been done. This is table stakes if you want to talk about your EDA algorithm advancement. If this weren't coming out of Google, everybody would laugh it…

> Nothing about AlphaChip even reaches ordinary evidence. You reply is wildly confident and dismissive. If correct, why did Nature choose to publish?

As Markov claims Nature did not follow their own policy. Since Google’s results are only on their designs, no one can replicate them. Nature is single blind, so they probably didn’t want to turn down Jeff Dean so that they wouldn’t lose future business from Google.

Re: Jeff Dean responds to EDA industry about AlphaChip

#90

Post by gabegobblegoldi »

btilly wrote: post #49
bsder wrote: post #40

Earlier quoted context omitted.

> Why do a non-zero amount of people have seemingly religious beliefs about this topic on one side or the other? Because lots of engineers are being told by managers "Why aren't we using that tool?" and a bunch of engineers are stuck saying "Because it doesn't actually work." aka "Google is lying through their teeth." to which the response is "Oh, so you know better than Google?" to which the reponse is "Yeah, actual…

And do you believe that that is what's happening in this case? If you have personal experience with Jeff Dean et al that you're willing to share, I'd be interested in hearing about it. From where I'm sitting it looks like, "Google spent a fortune on deep learning, and got a small but real win. People who don't like Google failed to follow Google's recipe and got a large and easily replicated loss." It's not even clea…

In this case there were credible claims of fraud from Google insiders. See my comment above.
Post reply on HN