Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jeff Dean responds to EDA industry about AlphaChip

twitter.com

Post reply on HN
Print view View original

171–180 of 221 posts

Re: Jeff Dean responds to EDA industry about AlphaChip

#171

Post by sijnapq »

wholehog wrote: post #156
gabegobblegoldi wrote: post #135

Earlier quoted context omitted.

Looks like he aligned himself with the wrong folks here. He is a system builder at heart but not an expert in chip design or EDA. And also not really an ML researcher. Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. His focus on this project didn’t help with his case at Google. They moved all the important stuff away from him and gave it to Demis last year and…

> Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. Was the TPU physical design team also taken in? And also MediaTek? And also TF-Agents, which publicly said they re-produced the AlphaChip method and results exactly?

[dead]

Re: Jeff Dean responds to EDA industry about AlphaChip

#172

Post by sijnapq »

rowanG077 wrote: post #79
throwaway2037 wrote: post #66

Earlier quoted context omitted.

> Nothing about AlphaChip even reaches ordinary evidence. You reply is wildly confident and dismissive. If correct, why did Nature choose to publish?

Can you stop with this pure appeal to authority. Publishing in nature is not proof it works. It's only proof the paper has packaged the claim it works semi well.

[dead]

Re: Jeff Dean responds to EDA industry about AlphaChip

#173

Post by xpe »

11101010001100 wrote: post #120
xpe wrote: post #37

Earlier quoted context omitted.

If so, does this qualify as “snake oil”? What do you mean? Snake oil requires exaggeration and deception. Fair? If a paper / experiment is done with intellectual honesty, great! If it doesn’t make a big splash, fine.

The paper is more or less a dead end. If there is another name you want to call it, by all means.

/[01]{8,}/: I was hoping to have a conversation. This is why I asked questions. Any responses to them?

Looking up the thread, you can see the context. Many of us pushed back against vague claims that AlphaChip was "snake oil". Like good engineers, we split apart the problem into clearer concepts. The "snake oil" proponents did not offer compelling replies, did they? Instead, they retreated to irrelevant points that have no bearing on making sense of the "snake oil" claim.

Sometimes technical people forget to bring their "debugging" skills to bear on conversations. There is a metaphorical connection; good debuggers would disambiguate terms, decompose the problem, answer questions, find cruxes, synthesize, find clearer terms, generate alternative explanations, and so on.

Re: Jeff Dean responds to EDA industry about AlphaChip

#174

Post by otterley »

wholehog wrote: post #157
dogleg77 wrote: post #143

Earlier quoted context omitted.

That's an appeal to authority, and not an effective one. Jeff Dean doesn't have a good track record in chip design.

What are you even talking about? Jeff had a hand in TPU, which is so successful that all other AI companies are trying to clone this project and spin up their own efforts to make custom AI chips.

By what measure are TPUs “successful”? Where is your data coming from?

Re: Jeff Dean responds to EDA industry about AlphaChip

#175

Post by dogleg77 »

wholehog wrote: post #156
gabegobblegoldi wrote: post #135

Earlier quoted context omitted.

Looks like he aligned himself with the wrong folks here. He is a system builder at heart but not an expert in chip design or EDA. And also not really an ML researcher. Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. His focus on this project didn’t help with his case at Google. They moved all the important stuff away from him and gave it to Demis last year and…

> Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. Was the TPU physical design team also taken in? And also MediaTek? And also TF-Agents, which publicly said they re-produced the AlphaChip method and results exactly?

What did the TPU physical design team say about this publicly? Can you also point to a statement from MediaTek? (I've seen a quote in Google blog, but was unable to confirm it). Who in the TF-agents team has serious physical design background?

Re: Jeff Dean responds to EDA industry about AlphaChip

#176

Post by dogleg77 »

nemonemo wrote: post #136
vighneshiyer wrote: post #121

Earlier quoted context omitted.

Cadence in particular has been quite receptive to allowing academics and researchers to benchmark new algorithms against their tools. They have also been quite permissive with letting people publish TCL scripts for their tools ( https://github.com/TILOS-AI-Institute/MacroPlacement/tree/ma... ) that in theory should enable precise reproduction of results. From my knowledge, Cadence has been very permissive from 2022 o…

We're not just talking about academia—Google's AlphaChip has the potential to disrupt the balance of the EDA industry's duopoly. It seems unlikely that Google could easily secure the policy or license changes necessary to publish direct comparisons in this context. If publicizing comparisons of CMPs is as permissible as you suggest, have you seen a publication that directly compares a Cadence macro placement tool wit…

I am trying to understand what you mean here by potential to disrupt. AlphaChip addresses one out of hundreds of tasks in chip design. Macro placement is a part of mixed-size placement, which is handled just fine by existing tools, many academic tools, open-source tools, and Nvidia AutoDMP. Even if AlphaChip was commonly accepted as a breakthrough, there is no disruption here. Direct comparisons from the last 3 years show that AlphaChip is worse. Granted, Google is belittling these comparisons, but that's what you'd expect. In any case, evidence is evidence.

Re: Jeff Dean responds to EDA industry about AlphaChip

#177

Post by dogleg77 »

Jabbles wrote: post #98
vighneshiyer wrote: post #67

Earlier quoted context omitted.

You are correct. For commercial use, the GPUs used for training and fine-tuning aren't a problem financially. However, if we wanted to rigorously benchmark AlphaChip against simulated annealing or other floorplanning algorithms, we have to afford the same compute and runtime budget to each algorithm. With 16 GPUs running for 6 hours, you could explore a huge placement space using any algorithm, and it isn't clear if…

You're saying that if the other methods were given the equivalent amount of compute they might be able to perform as well as AlphaChip? Or at least that the comparison would be fairer? Are the other methods scalable in that way?

The Google internal paper by Chatterjee and the Cheng et al paper from UCSD made such comparisons with Simulated Annealing. The annealer in the Nature paper was easy to improve. When given the same time budget, the improved annealer produced better solutions than AlphaChip. When you give both more time, SA remains ahead. Just read published papers.

Re: Jeff Dean responds to EDA industry about AlphaChip

#178

Post by dogleg77 »

wholehog wrote: post #158
fxtentacle wrote: post #125

Earlier quoted context omitted.

It seems that Chatterjee - the bad guy in your linked article - is now suing Google because he thinks he got canned for pointing out that his boss - Jeff Dean mentioned in the article discussed here - was knowingly publishing fraudulent claims. "To be clear, we do NOT have evidence to believe that RL outperforms academic state-of—art and strongest commercial macro placers. The comparisons for the latter were done so…

You're linking to his amended complaint - his original complaint was thrown out because it alleged things like "Google's motto is don't be evil, but they were evil, thus defrauding me." According to a Google investigator's sworn statement, he admitted that he didn't have evidence to suspect the AlphaChip authors of fraud: "he stated that he suspected that the research being conducted by Goldie and Mirhoseini was frau…

Something is off with your source references. Chatterjee's amended complaint is a legal document filed under penalty of perjury, accepted by a US judge, available publicly, apparently not "thrown out". How does an earlier document figure into this? How do we know it was "thrown out" and for what reason? Obviously, a later document is what matters.

Also, you are using an unreviewed document from Google not published in any conference to counter published papers with specific results, primarily the Cheng et al paper. Jeff Dean did like that paper, so he can take it up with the conference and convince them to unpublish it. If he can't, maybe he is wrong.

Perhaps, you are biased toward Google, but why do think we should trust a document that was neither peer-reviewed nor published at a conference?

Re: Jeff Dean responds to EDA industry about AlphaChip

#179

Post by sangnoir »

solidasparagus wrote: post #118
menaerus wrote: post #96

Earlier quoted context omitted.

Yes, they even do at $1/GPU/hr. However, 8xH100 cluster at full utilization is ~8kWh of electricity and costs almost ~0.5M$. 16xH100 cluster is probably 2x of that. How many years before you break-even at ~24$/GPU/day income?

Who cares? That's someone else's problem. I just pay 2-3$/hr and the H100s are usable

You should care about counterparty risks. If your business model depends on unsustainable 3rd party prices powered by VC largesse and unrealizable dreams of dominance, the very least you can do is plan for the impending reckoning, after which GPU proces will be determined by costs.

Re: Jeff Dean responds to EDA industry about AlphaChip

#180

Post by wholehog »

pclmulqdq wrote: post #164
wholehog wrote: post #155

Earlier quoted context omitted.

See my comment above - the Nature authors already did this, and tried a huge hyperparameter sweep for SA, and RL still won. See appendix of the Nature article: rdcu.be/cmedX

I understand and have read the article. Running 80 experiments with a crude form of simulated annealing is at most 0.0000000001% of the effort that has been spent on making that kind of hill climb work well by traditional EDA vendors. That is also an in-sample comparison, where I would believe the Google thing pre-trained on Google chips would do well, while it might have a harder time with a chip designed by a third…

> it might have a harder time with a chip designed by a third party (further from its pre-training).

Then they could pre-train on chips that are in-distribution for that task.

See also section 3.1 of their response paper, where they describe a comparison against commercial autoplacers: https://arxiv.org/pdf/2411.10053

Post reply on HN