Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jeff Dean responds to EDA industry about AlphaChip

twitter.com

Post reply on HN
Print view View original

21–30 of 221 posts

Re: Jeff Dean responds to EDA industry about AlphaChip

#21

Post by griomnib »

twothreeone wrote: post #16
jsnell wrote: post #8

Earlier quoted context omitted.

It's a good thing that he didn't say that, then. The tweet just says that the reproduction attempt didn't didn't actually follow the original methodology. There is no claim that the authors of the replication attempt were "idiots" or anything similar, you just made that up. The obviously fallacious logic in "they couldn't replicate it ..., therefore it's replicable" is also a total fabrication on your part.

A Google Nature Paper has not been replicated for over 3 years, but I'm the one fabricating stuff :D Making a novel claim implies its *_claimed_ replicability. "You did not follow the steps" is calling them idiots. The only inference I made is that he's pressed to comment. He could have said nothing.. instead he's lashing out publicly, because other people were unable to replicate it. If there's no problem replicatin…

“You can only validate my results if you have an entire Google data center worth of compute available. Since you don’t, you can’t question us.”

Re: Jeff Dean responds to EDA industry about AlphaChip

#22

Post by jeffbee »

griomnib wrote: post #20
twothreeone wrote: post #2

I get why Jeff would be pressed to comment on this, given he's credited on basically all of "Google Brain" research output. But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. Sounds like the critics struck a nerve and there's no good way for him to refute the replication problem his research apparently exhibits.

I love how he’s claiming bias due to his critic’s employer. As though working for Google has no conflicts? A company that is desperately hyping each and every “me too” AI development to juice the stock price? Jeff drank so much kool aid he forget what water is.

He's criticizing Markov for not disclosing the conflict, not for the conflict itself. Hiding your affiliation in a scientific publication is far outside the norms of science, and they should be criticized for that. The publication we are discussing — "That Chip Has Sailed" — dismisses Markov in a few paragraph and spends the bulk its arguments on Cheng.

Re: Jeff Dean responds to EDA industry about AlphaChip

#23

Post by griomnib »

xpe wrote: post #17
bsder wrote: post #4

The fact that the EDA companies are garbage in no way mitigates the fact that Google continues to peddle unsubstantiated snake oil. This is easy to debunk from the Google side: release a tool. If you don't want to release a tool, then it's unsubstantiated and you don't get to publish. Simple. That having been said: 1) None of these "AI" tools have yet demonstrated the ability to classify "This is datapath", "This is…

> Google continues to peddle unsubstantiated snake oil I read your comment, but I'm not following -- or maybe I disagree with it -- I'm not sure yet. "Snake oil" is an emotionally loaded term that raises the temperature of the conversation. That usually makes having a conversation harder. From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. Agree? Are you claiming that AlphaChip is not?…

They have literally been caught faking AI demos, they brought distrust on themselves.

Re: Jeff Dean responds to EDA industry about AlphaChip

#24

Post by twothreeone »

xpe wrote: post #13
twothreeone wrote: post #9

Earlier quoted context omitted.

It's not a value judgement, just doesn't help his case at all. He'd need to counter the replication problem, but apparently that's not an option. Instead, he's making people who were unable to replicate it look bad, which actually strengthens their criticism.

> But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. > It's not a value judgement, just doesn't help his case at all. Calling it "bullying" looks like a value judgment to me. Am I missing something? To me, Dean's response is quite sensible, particularly given his claims the other papers made serious mistakes and have potential conflicts of inter…

I'm not saying "Bullying is bad and bullies are bad people", that would be a value judgement. I'm saying bullying is the strictly worse strategy for strengthening his paper's claims in this scenario. The better strategy would be to foster an environment in which people can easily replicate your claims.

Re: Jeff Dean responds to EDA industry about AlphaChip

#25

Post by griomnib »

jeffbee wrote: post #22
griomnib wrote: post #20

Earlier quoted context omitted.

I love how he’s claiming bias due to his critic’s employer. As though working for Google has no conflicts? A company that is desperately hyping each and every “me too” AI development to juice the stock price? Jeff drank so much kool aid he forget what water is.

He's criticizing Markov for not disclosing the conflict, not for the conflict itself. Hiding your affiliation in a scientific publication is far outside the norms of science, and they should be criticized for that. The publication we are discussing — "That Chip Has Sailed" — dismisses Markov in a few paragraph and spends the bulk its arguments on Cheng.

I know the norms of science, I also know norms of present day Google. Nobody from Google should have the gall to accuse others of anything.

Re: Jeff Dean responds to EDA industry about AlphaChip

#26

Post by 11101010001100 »

xpe wrote: post #17
bsder wrote: post #4

The fact that the EDA companies are garbage in no way mitigates the fact that Google continues to peddle unsubstantiated snake oil. This is easy to debunk from the Google side: release a tool. If you don't want to release a tool, then it's unsubstantiated and you don't get to publish. Simple. That having been said: 1) None of these "AI" tools have yet demonstrated the ability to classify "This is datapath", "This is…

> Google continues to peddle unsubstantiated snake oil I read your comment, but I'm not following -- or maybe I disagree with it -- I'm not sure yet. "Snake oil" is an emotionally loaded term that raises the temperature of the conversation. That usually makes having a conversation harder. From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. Agree? Are you claiming that AlphaChip is not?…

Their material discovery paper turned out to have negligible significance.

Re: Jeff Dean responds to EDA industry about AlphaChip

#27

Post by jeffbee »

griomnib wrote: post #21
twothreeone wrote: post #16

Earlier quoted context omitted.

A Google Nature Paper has not been replicated for over 3 years, but I'm the one fabricating stuff :D Making a novel claim implies its *_claimed_ replicability. "You did not follow the steps" is calling them idiots. The only inference I made is that he's pressed to comment. He could have said nothing.. instead he's lashing out publicly, because other people were unable to replicate it. If there's no problem replicatin…

“You can only validate my results if you have an entire Google data center worth of compute available. Since you don’t, you can’t question us.”

We're actually talking about the difference between Cheng using 8 GPUs and 2 CPUs while Google used 16 GPUs and 40 CPUs. These are under-your-desk levels of resources. Cheng et al authors are all affiliated with UCSD which owns the Expanse supercomputer which is orders of magnitude larger than what you would need to reproduce the original work. Cheng et al does not explain why they used fewer resources.

Re: Jeff Dean responds to EDA industry about AlphaChip

#28

Post by griomnib »

jeffbee wrote: post #27
griomnib wrote: post #21

Earlier quoted context omitted.

“You can only validate my results if you have an entire Google data center worth of compute available. Since you don’t, you can’t question us.”

We're actually talking about the difference between Cheng using 8 GPUs and 2 CPUs while Google used 16 GPUs and 40 CPUs. These are under-your-desk levels of resources. Cheng et al authors are all affiliated with UCSD which owns the Expanse supercomputer which is orders of magnitude larger than what you would need to reproduce the original work. Cheng et al does not explain why they used fewer resources.

That’s a fair complaint then.

Re: Jeff Dean responds to EDA industry about AlphaChip

#29

Post by skybrian »

twothreeone wrote: post #18
skybrian wrote: post #12

Earlier quoted context omitted.

I don't know how you rebut a flawed paper without making its authors look bad? That would be a general-purpose argument against criticizing papers. Actually, people should criticize flawed papers. That's how science works! When you publish scientific papers, you should expect criticism if there's something that doesn't look right. The only way to avoid that is to get critical feedback before publishing the paper, and…

The situation here is different though.. If I'm making an existence claim by demonstrating a constructive argument and then being criticized for it, the most effective response to that critique would be a second, alternative construction, not attacking the critic's argument. After all, I'm the one claiming existence.. the burden of proof is on me, not my critics.

I don't know which argument is more constructive, though? Both teams reported what they did. They got different results. Figuring out why is the next step, and pointing out that they did different things seems useful.

Though, the broader question is how useful the results of the original paper are to other people who might do the same thing.

Re: Jeff Dean responds to EDA industry about AlphaChip

#30

Post by xpe »

twothreeone wrote: post #24
xpe wrote: post #13

Earlier quoted context omitted.

> But saying "they couldn't replicate it because they're idiots, therefore it's replicable" is not a rebuttal, just bullying. > It's not a value judgement, just doesn't help his case at all. Calling it "bullying" looks like a value judgment to me. Am I missing something? To me, Dean's response is quite sensible, particularly given his claims the other papers made serious mistakes and have potential conflicts of inter…

I'm not saying "Bullying is bad and bullies are bad people", that would be a value judgement. I'm saying bullying is the strictly worse strategy for strengthening his paper's claims in this scenario. The better strategy would be to foster an environment in which people can easily replicate your claims.

I think for most people the word “bullying” has a value judgment built-in.
Post reply on HN