Live data from Hacker News

Jeff Dean responds to EDA industry about AlphaChip

twitter.com

121–130 of 221 posts

Re: Jeff Dean responds to EDA industry about AlphaChip

#121

Earlier quoted context omitted.

So I'm not sure what Google is referring to here. As you can see in the ISPD paper ( https://vlsicad.ucsd.edu/Publications/Conferences/396/c396.p... ) on page 5, they openly compare Cadence CMP with AutoDMP and other algorithims quantitatively. The only obfuscation is with the proprietary GF12 technology, where they can't provide absolute numbers, but only relative ones. Comparison against commercial tools is actuall…

The UCSD paper says "We thank ... colleagues at Cadence and Synopsys for policy changes that permit our methods and results to be reproducible and sharable in the open, toward advancement of research in the field." This suggests that there may have been policies restricting publication prior to this work. It would be intriguing to see if future research on AlphaChip could receive a similar endorsement or support from…

Cadence in particular has been quite receptive to allowing academics and researchers to benchmark new algorithms against their tools. They have also been quite permissive with letting people publish TCL scripts for their tools (https://github.com/TILOS-AI-Institute/MacroPlacement/tree/ma...) that in theory should enable precise reproduction of results. From my knowledge, Cadence has been very permissive from 2022 onwards, so while Google's objections to publishing data from CMP may have been valid when the Nature paper was published, they are no longer valid today.

Re: Jeff Dean responds to EDA industry about AlphaChip

#122
post #98

Earlier quoted context omitted.

You are correct. For commercial use, the GPUs used for training and fine-tuning aren't a problem financially. However, if we wanted to rigorously benchmark AlphaChip against simulated annealing or other floorplanning algorithms, we have to afford the same compute and runtime budget to each algorithm. With 16 GPUs running for 6 hours, you could explore a huge placement space using any algorithm, and it isn't clear if…

You're saying that if the other methods were given the equivalent amount of compute they might be able to perform as well as AlphaChip? Or at least that the comparison would be fairer? Are the other methods scalable in that way?

Existing mixed-placement algorithms depend on hyperparameters, heuristics, and initial states / randomness. If afforded more compute resources, they can explore a much wider space and in theory come up with better solutions. Some algorithms like simulated annealing are easy to modify to exploit arbitrarily more compute resources. Indeed, I believe the comparison of AlphaChip to alternatives would be fairer if compute resources and allowed runtime were matched.

In fact, existing algorithms such as naive simulated annealing can be easily augmented with ML (e.g. using state embeddings to optimize hyperparameters for a given problem instance, or using a regression model to fine-tune proxy costs to better correlate with final QoR). Indeed, I strongly suspect commercial CAD software is already applying ML in many ways for mixed-placement and other CAD algorithms. The criticism against AlphaChip isn't about rejecting any application of ML to EDA CAD algorithms, but rather the particular formulation they used and objections to their reported results / comparisons.

Re: Jeff Dean responds to EDA industry about AlphaChip

#123

It's ridiculous how expensive the wrong hire can be https://www.wired.com/story/google-brain-ai-researcher-fired...

It is indeed a big deal to hire people who will commit or contrive at fraud: academic, financial, or otherwise.

But the best (probably only) way to put downward pressure on that is via internal incentives, controls, and culture. You push hard enough for such percent per cadence with no upper bound and graduate the folks who reliably deliver it without checking if the win was there to begin with? This is scale-invariant: it could be in a pod, a department, a company, a hedge fund that owns much of those companies, a fund of those funds, the federal government.

Sooner or later your leadership is substantially penetrated by the unscrupulous. We see this in academia with the spate of scandals around publications. We see this in finance with, who can even count that high anymore. You see Holmes and SBF in prison but the folks they funded still at the apex of relevance and everyone from that clique? Everyone who didn’t just fall of a turnip truck knows has carried that ideology with them and has better lawyers now.

There’s an old saw that a “fish rots from the head”. We can’t look at every manner of shadiness and constant scandal from the iconic leaders of our STEM industry and say “good for them, they outsmarted the system” and expect any result other than a broad-spectrum attack on any honest, fair, equitable status quo.

We all voted with our feet (and I did my share of that too before I quit in disgust) for a “might makes right” quasi-religious system of ideals, known variously as Objectivism, Effective Altruism, and Capitalism (of which it is no kind). We shouldn’t be surprised that everything is kind of tarnished sticky now.

The answer today? I don’t know. Work for the less bad as opposed to more bad companies, speak out at least anonymously about abuses, listen to the leaders speak in interviews and scrutinize it. I’m open to suggestions.

Re: Jeff Dean responds to EDA industry about AlphaChip

#124
I’ve not followed this story at all, and have no idea what is true or not, but generally when people use a boatload of adjectives which serve no purpose but to skew opinion, I assume they are not being honest. Using certain words to describe a situation does not make the situation what the author is saying, and if it is as they say, then the actual content should speak for itself.

For instance:

> Much of this unfounded skepticism is driven by a deeply flawed non-peer-reviewed publication by Cheng et al. that claimed to replicate our approach but failed to follow our methodology in major ways. In particular the authors did no pre-training (despite pre-training being mentioned 37 times in our Nature article),

This could easily be written more succinctly, and with less bias, as:

> Much of this skepticism is driven by a publication by Cheng et al. that claimed to replicate our approach but failed to follow our methodology in major ways. In particular the authors did no pre-training,

Calling the skepticism unfounded or deeply flawed does not make it so, and pointing out that a particular publication is not peer reviewed does not make its contents false. The authors would be better served by maintaining a more neutral tone rather than coming off accusatory and heavily biased.

Re: Jeff Dean responds to EDA industry about AlphaChip

#125

It's ridiculous how expensive the wrong hire can be https://www.wired.com/story/google-brain-ai-researcher-fired...

It seems that Chatterjee - the bad guy in your linked article - is now suing Google because he thinks he got canned for pointing out that his boss - Jeff Dean mentioned in the article discussed here - was knowingly publishing fraudulent claims.

"To be clear, we do NOT have evidence to believe that RL outperforms academic state-of—art and strongest commercial macro placers. The comparisons for the latter were done so poorly that in many cases the commercial tool failed to run due to installation issues." and that's supposedly a screenshot from an internal presentation done by Jeff Dean.

https://regmedia.co.uk/2023/03/26/satrajit_vs_google.pdf

As an outsider, I find it very difficult to judge if Chatterjee was a bad and expensive hire (because he suppressed good results by coworkers) or if he was a very valuable employee (because he tried to prevent publishing false statements).

Re: Jeff Dean responds to EDA industry about AlphaChip

#126
post #116
post #115

Earlier quoted context omitted.

So you're saying Head of AI of Google of Jeff can't choose a better venue? He's not the first Jeffery with a lot of power who doesn't care.

> So you're saying Head of AI of Google of Jeff can't choose a better venue? Phrasing it this way isn't useful. Talking about choice in the abstract doesn't help with a game-theoretic analysis. You need costs and benefits too. There are many people who face something like a prisoner's dilemma (on Twitter, for example). We could assess the cost-benefit of a particular person leaving Twitter. We could even judge them a…

Yes, but the game matrix is not that simple. There's a whole gamut of possible actions between defect and sleep with Elon.

Cross-posting to a Mastodon account is not that hard.

I look at this from two viewpoints. One is that it's good that he spends most of this time and energy doing research/management and not getting bogged down in culture war stuff. The other is that those who have all this power ought to wield it a tiny tiny bit more responsibly. (IMHO social influence of the elites/leaders/cool-kids are also among those leverage points you speak of.)

Also, I'm not blaming him. I don't think it's morally wrong to use X. (I think it's mentally harmful, but X is not unique in this. Though character limit does select for "no u" type messages.) I'm at best cynically musing about the claimed helplessness of Jeff Dean with regards to finding a forum.

Re: Jeff Dean responds to EDA industry about AlphaChip

#127

I don't get. Why isn't the model open if it works? If it isn't this is just a fart in the wind. If it is the findings should be straightforward to replicate.

Yes, the community should force Nature to up its standards or ditch it. Software replication should be trivial in this day and age.

Re: Jeff Dean responds to EDA industry about AlphaChip

#128

Curious why there's so much emotion and unpleasantness in this dispute? How did it evolve from the boring academic argument about benchmarks, significance, etc to a battle of personal attacks?

A lot of people work on non-AI implementations

This is a big part of the reason. But it behooves us to ask why a key innovation in a field (and I trust Jeff Dean that this is one, I’ve never seen any reason to doubt either his integrity or ability) should produce such a reaction. What could make people act not just chagrined that their approach wasn’t the end state, but as though it was existential to discredit such an innovation?

Surely all of the people who did the work that the innovation rests on should be confident they will be relevant, involved, comfortable, and safe in the post-innovation world?

And yet it’s not clear they should feel this way. Luddism seems an unfounded ideology over the scope of history since the origin of the term. But over the period since “AI” entered the public discussion at the current level? Almost two years exactly? Making the Luddite agenda credible has seemed a ubiquitous talking point.

Over that time frame technical people have been laid off in staggering numbers, a steadily-shrinking number of employers have been slashing headcount and posting EPS beats, and “AI” has been mentioned in every breath. It’s so extreme that even sophisticated knowledge of the kinds of subject matter that goes into AlphaChip is (allegedly) useless without access to the Hopper FLOPs.

If the AI Cartel was a little less rapacious, people might be a little more open to embracing the AI revolution.

Re: Jeff Dean responds to EDA industry about AlphaChip

#129
At this point in time, why wouldn't we give at least benefit of the doubt to Jeff Dean immediately? His track record is second to none, and he's still going strong. Has something happened that cast a shadow on him? Sometimes it is the messenger that brings in the weight.

Re: Jeff Dean responds to EDA industry about AlphaChip

#130

Additional context: Jeff Dean has been accused of fraud and misconduct in AlphaChip. https://regmedia.co.uk/2023/03/26/satrajit_vs_google.pdf

the link is for a wrongful termination lawsuit, related to the fraud but not a case for the fraud itself. settled may 2024
Post reply on HN