Deepmind is producing science vapourware while OpenAI is changing the world
[dead]
How AlphaChip transformed computer chip design
141–150 of 215 posts
Re: How AlphaChip transformed computer chip design
#142I must be old because first thing I thought reading AlphaChip was why is deepmind talking about chips in DEC Alpha :-) https://en.wikipedia.org/wiki/DEC_Alpha .
Same!
Re: How AlphaChip transformed computer chip design
#143Re: How AlphaChip transformed computer chip design
#144How far are we from memory-based computing going from research into competitive products? I get the impression that we are already well passed the point where it makes sense to invest very aggressively to scale up experiments with things like memristors. Because they are talking about how many new nuclear reactors they are going to need just for the AI datacenters.
The cognitive mismatch between Von Neumann's folly and other compute architectures is vast. He slowed down the ENIAC by 66% when he got ahold of it.
We're in the timeline that took the wrong path. The other world has isolinear memory, which can be used for compute, or as memory, down to the LUT level. Everything runs at a consistent speed, and hardware faults LUTs can be routed around easily.
Re: How AlphaChip transformed computer chip design
#145Earlier quoted context omitted.
Why does pretraining or not matter in the ISPD 2023 paper? The circuit_training repo, as noted in the rebuttal of the rebuttal by the ISPD 2023 paper authors, claims training from scratch is "comparable or better" than fine-tuning the pre-trained model. So no matter your opinion on the importance of the pretraining step, this result isn't replicable, at which point the ball is in Google's court to release code/checkp…
The quick-start guide in the repo that said you don't have to pre-train for the sample test case, meaning that you can validate your setup without pre-training. That does not mean you don't need to pre-train! Again, the paper talks at length about the importance of pre-training.
That does not mean you need to pre-train either. Common sense, no?
Re: How AlphaChip transformed computer chip design
#146Earlier quoted context omitted.
From what I saw in the rebuttal papers, the Google cost-function is wirelength based. You can still get good TNS from that if your timing is very simplistic -- or if you choose your benchmark carefully.
They optimize using a fast heuristic based on wirelength, congestion, and density, but they evaluate with full P&R. It is definitely interesting that they get good timing without explicitly including it in their reward function!
Yeah; it means the heuristic they use is a good one
Re: How AlphaChip transformed computer chip design
#147This work from Google (original Nature paper: https://www.nature.com/articles/s41586-021-03544-w ) has been credibly criticized by several researchers in the EDA CAD discipline. These papers are of interest: - A rebuttal by a researcher within Google who wrote this at the same time as the "AlphaChip" work was going on ("Stronger Baselines for Evaluating Deep Reinforcement Learning in Chip Placement"): http://47.190.8…
To be fair, some of these criticisms are a few years old. Which normally would be fair game, but the progress in AI has been breakneck. Criticism of other AI tech from 2021 or 2022 are pretty dated today.
Dated or not, if half of the criticisms are right, the original paper may need to be retracted. No progress on RL for chip design was published by Google since 2022, as far as I can tell. So, it looks like most if not all criticisms remain valid.
Re: How AlphaChip transformed computer chip design
#148Why do they keep saying "superhuman"? Algorithms are used for these tasks, humans aren't laying out trillions of transistors by hand.
I read the paper. Superhuman is a metric they defined in the paper which has to do with how long it takes a human to do certain tasks.
Does this make any sense, really? - Define some common words and then let the media run wild with them. How about we redefine "better" and "revolutionize"? Oh, wait, I think people are doing that already...
Re: How AlphaChip transformed computer chip design
#149So AI designing it's own chips. Now that is moving towards exponential growth. Like at the end of "Colossus" the movie. Forget LLM's. What DeepMind is doing seems more like how an AI will rule, in the world. Building real world models, and applying game logic like winning. LLM's will just be the text/voice interface to what DeepMind is building.
I can tell you get excited by SciFi, that's where Google's work belongs - people have been unable to reproduce it outside Google by a long shot.
Re: How AlphaChip transformed computer chip design
#150Earlier quoted context omitted.
Oh, man... this is the same old stuff from the 2023 Anna Goldie statement (is this Anna Goldie's comment?). This was all addressed by Kahng in 2023 - no valid criticisms. Where do I start? Kahng's ISPD 2023 paper is not in dispute - no established experts objected to it. The Nature paper is in dispute. Dozens of experts objected to it: Kahng, Cheng, Markov, Madden, Lienig, Swartz objected publically. The fact that Ka…
> Kahng is the most prominent active researcher in this field. If anyone knows this stuff, it's Kahng. This is written as a textbook example logical fallacy of appeal to authority.
[flagged]