Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How AlphaChip transformed computer chip design

deepmind.google

Post reply on HN
Print view View original

81–90 of 215 posts

Re: How AlphaChip transformed computer chip design

#81

Post by clickwiseorange »

negativeonehalf wrote: post #71
vighneshiyer wrote: post #41

This work from Google (original Nature paper: https://www.nature.com/articles/s41586-021-03544-w ) has been credibly criticized by several researchers in the EDA CAD discipline. These papers are of interest: - A rebuttal by a researcher within Google who wrote this at the same time as the "AlphaChip" work was going on ("Stronger Baselines for Evaluating Deep Reinforcement Learning in Chip Placement"): http://47.190.8…

FD: I have been following this whole thing for a while, and know personally a number of the people involved. The AlphaChip authors address criticism in their addendum, and in a prior statement from the co-lead authors: https://www.nature.com/articles/s41586-024-08032-5 , https://www.annagoldie.com/home/statement - The 2023 ISPD paper didn't pre-train at all. This means no learning from experience, for a learning-base…

Oh, man... this is the same old stuff from the 2023 Anna Goldie statement (is this Anna Goldie's comment?). This was all addressed by Kahng in 2023 - no valid criticisms. Where do I start?

Kahng's ISPD 2023 paper is not in dispute - no established experts objected to it. The Nature paper is in dispute. Dozens of experts objected to it: Kahng, Cheng, Markov, Madden, Lienig, Swartz objected publically.

The fact that Kahng's paper was invited doesn't mean it wasn't peer reviewed. I checked with ISPD chairs in 2023 - Kahng's paper was thoroughly reviewed and went through multiple rounds of comments. Do you accept it now? Would you accept peer-reviewed versions of other papers?

Kahng is the most prominent active researcher in this field. If anyone knows this stuff, it's Kahng. There were also five other authors in that paper, including another celebrated professor, Cheng.

The pre-training thing was disclaimed in the Google release. No code, data or instructions for pretraining were given by Google for years. The instructions said clearly: you can get results comparable to Nature without pre-training.

The "much older technology" is also a bogus issue because the HPWL scales linearly and is reported by all commercial tools. Rectangles are rectangles. This is textbook material. But Kahng etc al prepared some very fresh examples, including NVDLA, with two recent technologies. Guess what, RL did poorly on those. Are you accepting this result?

The bit about financial incentives and open-source is blatantly bogus, as Kahng leads OpenROAD - the main open-source EDA framework. He is not employed by any EDA companies. It is Google who has huge incentives here, see Demis Hassabis tweet "our chips are so good...".

The "Stronger Baselines" matched compute resources exactly. Kahng and his coauthors performed fair comparisons between annealing and RL, giving the same resources to each. Giving greater resources is unlikely to change results. This was thoroughly addressed in Kahng's FAQ - if you only could read that.

The resources used by Google were huge. Cadence tools in Kahng's paper ran hundreds times faster and produced better results. That is as conclusive as it gets.

It doesn't take a Ph.D. to understand fair comparisons.

Re: How AlphaChip transformed computer chip design

#82

Post by ilaksh »

sroussey wrote: post #57
ilaksh wrote: post #22

How far are we from memory-based computing going from research into competitive products? I get the impression that we are already well passed the point where it makes sense to invest very aggressively to scale up experiments with things like memristors. Because they are talking about how many new nuclear reactors they are going to need just for the AI datacenters.

The problem is that the competition (our current von neumann architecture) has billions of dollars of R&D per year invested. Better architectures without the yearly investment train will no longer be better quite quickly. You would need to be 100x to 1000x better in order to pull the investment train onto your tracks. Don’t has been impossible for decades. Even so, I think we will see such a change in my lifetime. AI…

I think it's just ignorance and timidity on the part of investors. Memristor or memory-computing startups are surely the next trend in investing within a few years.

I don't think it's necessarily demand or any particular calculation that makes things happen. I think people including investors are just herd animals. They aren't enthusiastic until they see the herd moving and then they want in.

Re: How AlphaChip transformed computer chip design

#83

Post by bsder »

vighneshiyer wrote: post #41

This work from Google (original Nature paper: https://www.nature.com/articles/s41586-021-03544-w ) has been credibly criticized by several researchers in the EDA CAD discipline. These papers are of interest: - A rebuttal by a researcher within Google who wrote this at the same time as the "AlphaChip" work was going on ("Stronger Baselines for Evaluating Deep Reinforcement Learning in Chip Placement"): http://47.190.8…

EDA claims in the digital domain are fairly easy to evaluate. Look at the picture of the layout.

When you see a chip that has the datapath identified and laid out properly by a computer algorithm, you've got something. If not, it's vapor.

So, if your layout still looks like a random rat's nest? Nope.

If even a random person can see that your layout actually follows the obvious symmetric patterns from bit 0 to bit 63, maybe you've got something worth looking at.

Analog/RF is a little tougher to evaluate because the smaller number of building blocks means you can use Moore's Law to brute force things much more exhaustively, but if things "looks pretty" then you've got something. If it looks weird, you don't.

Re: How AlphaChip transformed computer chip design

#85

Post by glitchc »

bsder wrote: post #83
vighneshiyer wrote: post #41

This work from Google (original Nature paper: https://www.nature.com/articles/s41586-021-03544-w ) has been credibly criticized by several researchers in the EDA CAD discipline. These papers are of interest: - A rebuttal by a researcher within Google who wrote this at the same time as the "AlphaChip" work was going on ("Stronger Baselines for Evaluating Deep Reinforcement Learning in Chip Placement"): http://47.190.8…

EDA claims in the digital domain are fairly easy to evaluate. Look at the picture of the layout. When you see a chip that has the datapath identified and laid out properly by a computer algorithm, you've got something. If not, it's vapor. So, if your layout still looks like a random rat's nest? Nope. If even a random person can see that your layout actually follows the obvious symmetric patterns from bit 0 to bit 63,…

That doesn't mean the fabricated netlist doesn't work. I'm not supporting Google by any means, but the test should be: Does it fabricate and function as intended? If not, clearly gibberish. If so, we now have computers building computers, which is one step closer to SkyNet. The truth is probably somewhere in between. But even if some of the samples, with the terrible layouts, are actually functional, then we might learn something new. Maybe the gibberish design has reduced crosstalk, which would be fascinating.

Re: How AlphaChip transformed computer chip design

#86

Post by negativeonehalf »

clickwiseorange wrote: post #81
negativeonehalf wrote: post #71

Earlier quoted context omitted.

FD: I have been following this whole thing for a while, and know personally a number of the people involved. The AlphaChip authors address criticism in their addendum, and in a prior statement from the co-lead authors: https://www.nature.com/articles/s41586-024-08032-5 , https://www.annagoldie.com/home/statement - The 2023 ISPD paper didn't pre-train at all. This means no learning from experience, for a learning-base…

Oh, man... this is the same old stuff from the 2023 Anna Goldie statement (is this Anna Goldie's comment?). This was all addressed by Kahng in 2023 - no valid criticisms. Where do I start? Kahng's ISPD 2023 paper is not in dispute - no established experts objected to it. The Nature paper is in dispute. Dozens of experts objected to it: Kahng, Cheng, Markov, Madden, Lienig, Swartz objected publically. The fact that Ka…

For AlphaChip, pre-training is just training. You train, and save the weights in between. This has always been supported by the Google's open-source repository. I've read Kahng's FAQ, and he fails to address this, which is unsurprising, because there's simply no excuse for cutting out pre-training for a learning-based method. In his setup, every time AlphaChip sees a new chip, he re-randomizes the weights and makes it learn from scratch. This is obviously a terrible move.

HPWL (half-perimeter wirelength) is an approximation of wirelength, which is only one component of the chip floorplanning objective function. It is relatively easy to crunch all the components together and optimize HPWL --- minimizing actual wirelength while avoiding congestion issues is much harder.

Simulated annealing is good at quickly converging on a bad solution to the problem, with relatively little compute. So what? We aren't compute-limited here. Chip design is a lengthy, expensive process where even a few-percent wirelength reduction can be worth millions of dollars. What matters is the end result, and ML has SA beat.

(As for conflict of interest, my understanding is Cadence has been funding Kahng's lab for years, and Markov's LinkedIn says he works for Synopsis. Meanwhile, Google has released a free, open-source tool.)

Re: How AlphaChip transformed computer chip design

#87

Post by jeffbee »

QuadrupleA wrote: post #78

How good are TPUs in comparison with state of the art Nvidia datacenter GPUs, or Groq's ASICs? Per watt, per chip, total cost, etc.? Is there any published data?

MLPerf is a good place to start. The only problem is you don't have any verifiable information about TPU energy consumption. https://mlcommons.org/benchmarks/inference-datacenter/

Re: How AlphaChip transformed computer chip design

#88

Post by wslh »

QuadrupleA wrote: post #78

How good are TPUs in comparison with state of the art Nvidia datacenter GPUs, or Groq's ASICs? Per watt, per chip, total cost, etc.? Is there any published data?

I have some company notes from early 2024 which cannot be accurate but could help,

TPU v5e [1]: not available for purchase, only through GCP, storage=5B, LLM-Model=7B, efficiency=393TFLOP.

[1] https://cloud.google.com/tpu/docs/v5e

Re: How AlphaChip transformed computer chip design

#89

Post by ur-whale »

Seems to me the article is claiming a lot of things, but is very light on actual comparisons that matter to you and me, namely: how does one of those fabled AI-designed chop compare to their competition ?

For example, how much better are these latest gen TPU's when compared to NVidia's equivalent offering ?

Re: How AlphaChip transformed computer chip design

#90

Post by negativeonehalf »

Drunk_Engineer wrote: post #73
pfisherman wrote: post #17

Questions for those in the know about chip design. How are they measuring the quality of a chip design? Does the metric that Google is reporting make sense? Or is it just something to make themselves look good? Without knowing much, my guess is that “quality” of a chip design is multifaceted and heavily dependent on the use case. That is the ideal chip for a data center would look very different from those for a mobi…

I have not read the latest paper, but their previous work was really unclear about metrics being used. Researchers trying to replicate results had a hard time getting reliable details/benchmarks out of Google. Also, my recollection is that Google did not even compute timing, just wirelength and congestion; i.e. extremely primitive metrics. Floorplanning/placement/synthesis is a billion dollar industry, so if their ap…

The original paper reports P&R metrics (WNS, TNS, area, power, wirelength, horizontal congestion, vertical congestion) - https://www.nature.com/articles/s41586-021-03544-w

(no paywall): https://www.cl.cam.ac.uk/~ey204/teaching/ACS/R244_2021_2022/...

Post reply on HN