Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jeff Dean responds to EDA industry about AlphaChip

twitter.com

Post reply on HN
Print view View original

31–40 of 221 posts

Re: Jeff Dean responds to EDA industry about AlphaChip

#31

Post by xpe »

twothreeone wrote: post #24
xpe wrote: post #13

Earlier quoted context omitted.

> But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. > It's not a value judgement, just doesn't help his case at all. Calling it "bullying" looks like a value judgment to me. Am I missing something? To me, Dean's response is quite sensible, particularly given his claims the other papers made serious mistakes and have potential conflicts of inter…

I'm not saying "Bullying is bad and bullies are bad people", that would be a value judgement. I'm saying bullying is the strictly worse strategy for strengthening his paper's claims in this scenario. The better strategy would be to foster an environment in which people can easily replicate your claims.

In a perfect world, making a paper easier to replicate has advantages, sure. (But it also has costs.)

Second, even a healthy environment can be undermined by lack of skills or resources, intellectual dishonesty, or conflicts of interest.

Re: Jeff Dean responds to EDA industry about AlphaChip

#32

Post by rajup »

griomnib wrote: post #23
xpe wrote: post #17

Earlier quoted context omitted.

> Google continues to peddle unsubstantiated snake oil I read your comment, but I'm not following -- or maybe I disagree with it -- I'm not sure yet. "Snake oil" is an emotionally loaded term that raises the temperature of the conversation. That usually makes having a conversation harder. From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. Agree? Are you claiming that AlphaChip is not?…

They have literally been caught faking AI demos, they brought distrust on themselves.

Really not sure how you’re conflating product demos which are known to be pie in the sky across the industry (not just Google) with peer reviewed research published in journals. Super basic distinction imho.

Re: Jeff Dean responds to EDA industry about AlphaChip

#33

Post by xpe »

twothreeone wrote: post #24
xpe wrote: post #13

Earlier quoted context omitted.

> But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. > It's not a value judgement, just doesn't help his case at all. Calling it "bullying" looks like a value judgment to me. Am I missing something? To me, Dean's response is quite sensible, particularly given his claims the other papers made serious mistakes and have potential conflicts of inter…

I'm not saying "Bullying is bad and bullies are bad people", that would be a value judgement. I'm saying bullying is the strictly worse strategy for strengthening his paper's claims in this scenario. The better strategy would be to foster an environment in which people can easily replicate your claims.

Are you suggesting Dean take a different approach in his response? Are you saying it was already too late given the environment? (I’m also not sure I know what you mean by environment here.)

Re: Jeff Dean responds to EDA industry about AlphaChip

#34

Post by johnfn »

twothreeone wrote: post #2

I get why Jeff would be pressed to comment on this, given he's credited on basically all of "Google Brain" research output. But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. Sounds like the critics struck a nerve and there's no good way for him to refute the replication problem his research apparently exhibits.

> "they couldn't replicate it because they're idiots, therefore it's replicable"

Where does it say that? Dean outlines explicit steps that the authors missed in the tweet.

Re: Jeff Dean responds to EDA industry about AlphaChip

#35

Post by iamnotafraid »

twothreeone wrote: post #2

I get why Jeff would be pressed to comment on this, given he's credited on basically all of "Google Brain" research output. But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. Sounds like the critics struck a nerve and there's no good way for him to refute the replication problem his research apparently exhibits.

Interesting point, I give u this

Re: Jeff Dean responds to EDA industry about AlphaChip

#36

Post by bsder »

joshuamorton wrote: post #7
bsder wrote: post #4

The fact that the EDA companies are garbage in no way mitigates the fact that Google continues to peddle unsubstantiated snake oil. This is easy to debunk from the Google side: release a tool. If you don't want to release a tool, then it's unsubstantiated and you don't get to publish. Simple. That having been said: 1) None of these "AI" tools have yet demonstrated the ability to classify "This is datapath", "This is…

As someone who has no skin in the game and is only loosely following this, there is a tool: https://github.com/google-research/circuit_training , the detractors claim to not be able to reproduce Google's results (what Dean is commenting on) with it, Google and 1-2 other companies claim to be using it internally to success (e.g. see the end of this article: https://deepmind.google/discover/blog/how-alphachip-transfor.…

There are benchmarks in this space. You can also bring your chip designs into the open and show what happens with different tools. You can run the algorithm on the placed designs that you sponsor for open source VLSI to show how much better they are.

None of this has been done. This is table stakes if you want to talk about your EDA algorithm advancement. If this weren't coming out of Google, everybody would laugh it out of the room (see what happened to a similar publication with similar claims from a Chinese source--everybody dismissed it out of hand--rightfully so even though that paper was MUCH better than anything Google has promulgated).

Extraordinary claims require extraordinary evidence. Nothing about AlphaChip even reaches ordinary evidence.

If they hadn't gotten a publication in Nature for effectively a failure, this would be way less contentious.

Re: Jeff Dean responds to EDA industry about AlphaChip

#37

Post by xpe »

11101010001100 wrote: post #26
xpe wrote: post #17

Earlier quoted context omitted.

> Google continues to peddle unsubstantiated snake oil I read your comment, but I'm not following -- or maybe I disagree with it -- I'm not sure yet. "Snake oil" is an emotionally loaded term that raises the temperature of the conversation. That usually makes having a conversation harder. From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. Agree? Are you claiming that AlphaChip is not?…

Their material discovery paper turned out to have negligible significance.

If so, does this qualify as “snake oil”? What do you mean? Snake oil requires exaggeration and deception. Fair?

If a paper / experiment is done with intellectual honesty, great! If it doesn’t make a big splash, fine.

Re: Jeff Dean responds to EDA industry about AlphaChip

#38

Post by vighneshiyer »

I have published an addendum to an article I wrote about AlphaChip (https://vighneshiyer.com/misc/ml-for-placement/) at the very bottom that addresses this rebuttal from Google and the AlphaChip algorithm in general.

In short, I think the Nature authors have made some reasonable criticisms regarding the training methodology employed by the ISPD authors, but the extreme compute cost and runtime of AlphaChip still makes it non-competitive with commercial autofloorplanners and AutoDMP. Regardless, I think the ISPD authors owe the Nature authors an even more rigorous study that addresses all their criticisms. Even if they just try to evaluate the pre-trained checkpoint that Google published, that would be a useful piece of data to add to the debate.

Re: Jeff Dean responds to EDA industry about AlphaChip

#39

Post by bsder »

xpe wrote: post #17
bsder wrote: post #4

The fact that the EDA companies are garbage in no way mitigates the fact that Google continues to peddle unsubstantiated snake oil. This is easy to debunk from the Google side: release a tool. If you don't want to release a tool, then it's unsubstantiated and you don't get to publish. Simple. That having been said: 1) None of these "AI" tools have yet demonstrated the ability to classify "This is datapath", "This is…

> Google continues to peddle unsubstantiated snake oil I read your comment, but I'm not following -- or maybe I disagree with it -- I'm not sure yet. "Snake oil" is an emotionally loaded term that raises the temperature of the conversation. That usually makes having a conversation harder. From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. Agree? Are you claiming that AlphaChip is not?…

> From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements.

These things you mentioned had obvious benchmarks that were easily surpassed by the appropriate "AI". The evidence that they were better wasn't just significant, it was obvious.

This leaves the fact that with what appears to be maximal cooking of the books, the only thing AlphaChip seems to be able to beat is human, manual placement and not anything algorithmic--even from many, many generations ago.

Trying to pass that off as a significant "advance" in a "scientific publication" borders on scientific fraud and should definitely be called out.

The problem here is that I am certain that this is wired to the career trajectories of "Very Important People(tm)" and the fact that it essentially failed miserably is simply not politically allowed.

If they want to lie, they can do that in press releases. If they want published in something reputable, they should have to be able to provide proper evidence for replication.

And, if they can't do that, well, that's an answer itself, no?

Re: Jeff Dean responds to EDA industry about AlphaChip

#40

Post by bsder »

bayarearefugee wrote: post #10
danpalmer wrote: post #6

Earlier quoted context omitted.

> But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying That's not an argument made in the linked tweet. His claim is "they couldn't replicate it because they didn't follow the steps", which seems like a very reasonable claim, regardless of the motivation behind making it.

At the end of the day my question is simply why does anyone care about the drama over this one way or another? Either the research is as much of a breakthrough as is claimed and Google is about to pull way ahead of all these other "idiots" who can't replicate their method even when it is described to them in detail, or the research is flawed and overblown and not as effective as claimed. This seems like exactly the s…

> Why do a non-zero amount of people have seemingly religious beliefs about this topic on one side or the other?

Because lots of engineers are being told by managers "Why aren't we using that tool?" and a bunch of engineers are stuck saying "Because it doesn't actually work." aka "Google is lying through their teeth." to which the response is "Oh, so you know better than Google?" to which the reponse is "Yeah, actually, I fucking do. Now piss off and let me finish timing closure this goddamn block that is already 6 weeks late."

Now can you understand why this is a bit contentious?

Marketing "exaggerations" from authority can cause huge amounts of grief.

In my little corner of the world, I had to sit and defend against the lies that a startup with famous designers were putting out about power consumption while we were designing similar chips in the space. I had to go toe to toe with Senior VPs over it and I had to stand my ground and defend my team who analyzed things dead on. All this occurred in spite of the fact that they had no silicon. In addition, I knew the famous designers involved would happily lie straight to your face having worked with them before and having been lied straight to my face and having had to clean up the mess when they left the company.

To be fair, it is also the only time I have had a Senior VP remember the kerfuffle and apologize when said startup finally delivered silicon and not only were the real numbers not what they claimed they weren't even close to the ones we were getting.

Post reply on HN