Earlier quoted context omitted.
[flagged]
> EQ Using a fantasy concept invented by a science journalist doesn't help your posts, you know. Protip: it's just empathy + regular intelligence.
yes you figured out what I meant good job
Jeff Dean responds to EDA industry about AlphaChip
101–110 of 221 posts
Re: Jeff Dean responds to EDA industry about AlphaChip
#102In the tweet Jeff Dean says that Cheng at al. failed to follow the steps required to replicate the work of the Google researchers. Specifically: > In particular the authors did no pre-training (despite pre-training being mentioned 37 times in our Nature article), robbing our learning-based method of its ability to learn from other chip designs But in the Circuit Training Google repo[1] they specifically say: > Our re…
Training from scratch could presumably mean including the new design attempts and old designs mixed in.
So no contradiction: pretrain on old designs then finetune on new design, vs train on everything mixed together throughout. Finetuning can cause catastrophic forgetting. Both could have better performance than not including old designs.
Re: Jeff Dean responds to EDA industry about AlphaChip
#103Earlier quoted context omitted.
You are correct. For commercial use, the GPUs used for training and fine-tuning aren't a problem financially. However, if we wanted to rigorously benchmark AlphaChip against simulated annealing or other floorplanning algorithms, we have to afford the same compute and runtime budget to each algorithm. With 16 GPUs running for 6 hours, you could explore a huge placement space using any algorithm, and it isn't clear if…
You're saying that if the other methods were given the equivalent amount of compute they might be able to perform as well as AlphaChip? Or at least that the comparison would be fairer? Are the other methods scalable in that way?
Yes, they are. The other approaches usually look like simulated annealing, which has several hyperparameters that control how much computing is used and improve results with more compute usage.
Re: Jeff Dean responds to EDA industry about AlphaChip
#104Re: Jeff Dean responds to EDA industry about AlphaChip
#105Earlier quoted context omitted.
> EDA companies are garbage I don't understand this comment. Can you please explain? Are they unethical? Or do they write poor software?
Yes and yes. EDA companies are gatekeeping monopolies. They absolutely abuse their monopoly position to extract huge chunks of money out of companies, and are pretty much single-handedly responsible for the fact that the hardware startup ecosystem is moribund compared to that of the software startup ecosystem. They have been horrible liars about performance and benchmarketing for decades. They dragged their feet mise…
> The EDA companies aren't quite Oracle--but they're not far off.
Agreed with most you mentioned but not about EDA companies are not worst than Oracle, at least Oracle is still supporting popular and useful open source projects namely MySQL, Virtualbox, etc.
What open-source design software these EDA companies are supporting currently although most of their software originated from open source EDA software from UC Berkeley, etc?
Re: Jeff Dean responds to EDA industry about AlphaChip
#106Curious why there's so much emotion and unpleasantness in this dispute? How did it evolve from the boring academic argument about benchmarks, significance, etc to a battle of personal attacks?
If you think this is unpleasant, you should see the environmentalists who try to take a poke at Jeff Dean on Twitter.
Well... I kinda expect some people to be overly emotional. But I just didn't expect this particular group of people to be that.
Re: Jeff Dean responds to EDA industry about AlphaChip
#107Earlier quoted context omitted.
Yes, they even do at $1/GPU/hr. However, 8xH100 cluster at full utilization is ~8kWh of electricity and costs almost ~0.5M$. 16xH100 cluster is probably 2x of that. How many years before you break-even at ~24$/GPU/day income?
7 https://www.google.com/search?q=0.5e6%2F8%2F24%2F365
Did you really not understand rethoric nature of my question and assumed that I can't do 1st grade primary school math?
Re: Jeff Dean responds to EDA industry about AlphaChip
#108Earlier quoted context omitted.
> Google continues to peddle unsubstantiated snake oil I read your comment, but I'm not following -- or maybe I disagree with it -- I'm not sure yet. "Snake oil" is an emotionally loaded term that raises the temperature of the conversation. That usually makes having a conversation harder. From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. Agree? Are you claiming that AlphaChip is not?…
> From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. These things you mentioned had obvious benchmarks that were easily surpassed by the appropriate "AI". The evidence that they were better wasn't just significant, it was obvious . This leaves the fact that with what appears to be maximal cooking of the books, the only thing AlphaChip seems to be able to beat is human, manual placemen…
> Trying to pass that off as a significant "advance" in a "scientific publication" borders on scientific fraud and should definitely be called out.
If true, your stated concerns with the AlphaChip paper -- selective benchmarking and potential overselling of results - reflect poor scientific practice and possible intellectual dishonesty. This does not constitute scientific fraud, which occurs when the underlying method/experiment/rules are faked.
If the paper has issues with how it positions and contextualizes its contribution, criticism is warranted, sure. But don't confuse this with "scientific fraud".
Some context: for as long as benchmark suites have existed, people rightly comment on which benchmarks should be included and how they should be weighted.
Re: Jeff Dean responds to EDA industry about AlphaChip
#109Earlier quoted context omitted.
Agreed, in particular on #2 Given infinite time and compute - maybe the approach is significantly better. But that’s just not practical. So unless you see dramatic shifts - no one is going to throw away proven results on your new approach because of the TTM penalty if it goes wrong. The EDA industry is (has to be) ultra conservative.
> The EDA industry is (has to be) ultra conservative. What is special about EDA that requires it to be more conservative?
Taping out a chip is an incredibly expensive (7-8 figure) fixed cost. If the chips that come out have too many bugs (say because your PD tools missed up some wiring for 1 in 10,000 blocks) then that money is gone. If you're Intel this is enough to make people doubt the health of your firm; if you're a startup, you're just done.
Re: Jeff Dean responds to EDA industry about AlphaChip
#110The fact that the EDA companies are garbage in no way mitigates the fact that Google continues to peddle unsubstantiated snake oil. This is easy to debunk from the Google side: release a tool. If you don't want to release a tool, then it's unsubstantiated and you don't get to publish. Simple. That having been said: 1) None of these "AI" tools have yet demonstrated the ability to classify "This is datapath", "This is…
> Google continues to peddle unsubstantiated snake oil I read your comment, but I'm not following -- or maybe I disagree with it -- I'm not sure yet. "Snake oil" is an emotionally loaded term that raises the temperature of the conversation. That usually makes having a conversation harder. From my point of view, AlphaGo, AlphaZero, AlphaFold were significant achievements. Agree? Are you claiming that AlphaChip is not?…
Well here’s one exaggeration that was pretty obvious to me straight away as a somewhat disinterested observer. In her status on X Anna Goldie says [1] “ AlphaChip was one of the first RL methods deployed to solve a real-world engineering problem”. This seems very clearly untrue- for example here’s a real-world engineering use of reinforcement learning by google AI themselves from 6 years ago [2] which if you use Anna Goldie’s own timeline is 2 years before alphachip.