Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jeff Dean responds to EDA industry about AlphaChip

twitter.com

Post reply on HN
Print view View original

151–160 of 221 posts

Re: Jeff Dean responds to EDA industry about AlphaChip

#151

Post by choppaface »

alsodumb wrote: post #139
choppaface wrote: post #47

Earlier quoted context omitted.

The result is minor AND Google spent a (relative) lot of money to achieve it (especially in the eyes of the new CFO). Jeff Dean is desperately trying to save the prestige of the research (in a very insular, Google-y way) because he wants to save the 2017-era economically-not-viable blue sky culture where Tensorflow & the TPU flourished and the transformer was born. But the reality is that Google’s core businesses are…

What makes you say TPU has zero chance against growing NVIDIA? If anything, now is the best time for TPU to grow and I'd say investing in TPU gave Google an edge. There is no other large scale LLM that was trained on anything but NVIDIA GPUs. Gemini is the only exception. Every big company is scrambling to make their own hardware in the AI era while Google already has it. Everyone I know who worked with TPUs loves ho…

Besides the many CAPEX-vs-OPEX tradeoffs that are completely unavailable due to not being able to buy physical TPU pods, there are inherent Google-y risks e.g. risk of the TPU product and/or support getting killed or fragmented / deprecated (very very common with Google), your data & traffic must also be locked in to Google’s pricing, and you must indefinitely put up with / negotiate with Google Cloud people (in my experience at multiple companies: worst customer support ever).

Google does indeed lock in their own ROI with deciding to not compete with AMD / Graphcore etc, but that also rooflines their total market. If they were to come up with a compelling Android-based Jetson-like edge product, and if demand for said product eclipses total GPU demand (robotics explosion?) then they might have a ramp to compete with NVidia. But the USB TPUs and phone accelerators today are just toys. And toys go to the Google graveyard, because Googlers don’t build gardens they treat everything like toys and throw them away when they get bored.

Re: Jeff Dean responds to EDA industry about AlphaChip

#152

Post by the-rc »

gabegobblegoldi wrote: post #135
Keyframe wrote: post #129

At this point in time, why wouldn't we give at least benefit of the doubt to Jeff Dean immediately? His track record is second to none, and he's still going strong. Has something happened that cast a shadow on him? Sometimes it is the messenger that brings in the weight.

Looks like he aligned himself with the wrong folks here. He is a system builder at heart but not an expert in chip design or EDA. And also not really an ML researcher. Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. His focus on this project didn’t help with his case at Google. They moved all the important stuff away from him and gave it to Demis last year and…

Not a ML researcher, too? He was working on neural networks in 1990. Last year he was under Research and now reports directly to Sundar. What do you know that we don't?

Re: Jeff Dean responds to EDA industry about AlphaChip

#153

Post by Analemma_ »

jiveturkey wrote: post #130
gabegobblegoldi wrote: post #85

Additional context: Jeff Dean has been accused of fraud and misconduct in AlphaChip. https://regmedia.co.uk/2023/03/26/satrajit_vs_google.pdf

the link is for a wrongful termination lawsuit, related to the fraud but not a case for the fraud itself. settled may 2024

"Settled" does not mean "Dean did nothing wrong". It means "Google paid the plaintiffs a lot of money so they'd stop saying publicly that Dean did something wrong", which is very different.

Re: Jeff Dean responds to EDA industry about AlphaChip

#154

Post by wholehog »

vighneshiyer wrote: post #67
wholehog wrote: post #50

Earlier quoted context omitted.

We're talking 16 GPUs for ~6 hrs for inference, and 48 hrs for pre-training. This is not an exorbitant amount of compute. A GPU costs $1-2/hr on the cloud market. So, ~$100-200 for inference, and ~$800-1600 for pre-training, which amortizes across chips. Cloud prices are an upper bound -- most CS labs will have way more than this available on premises. In an industry context, these costs are completely dwarfed by the…

You are correct. For commercial use, the GPUs used for training and fine-tuning aren't a problem financially. However, if we wanted to rigorously benchmark AlphaChip against simulated annealing or other floorplanning algorithms, we have to afford the same compute and runtime budget to each algorithm. With 16 GPUs running for 6 hours, you could explore a huge placement space using any algorithm, and it isn't clear if…

> However, if we wanted to rigorously benchmark AlphaChip against simulated annealing or other floorplanning algorithms, we have to afford the same compute and runtime budget to each algorithm.

The Nature authors already presented such a study in their appendix:

"To make comparisons fair, we ran 80 SA experiments sweeping different hyperparameters, including maximum temperature (10^−5, 3 × 10^−5, 5 × 10^−5, 7 × 10^-5, 10^−4, 2 × 10^−4, 5 × 10^−4, 10^−3), maximum SA episode length (5 × 10^4, 10^5) and seed (five different random seeds), and report the best results in terms of proxy wirelength and congestion costs in Extended Data Table 6"

Non-paywalled Nature article link: rdcu.be/cmedX

Re: Jeff Dean responds to EDA industry about AlphaChip

#155

Post by wholehog »

pclmulqdq wrote: post #103
Jabbles wrote: post #98

Earlier quoted context omitted.

You're saying that if the other methods were given the equivalent amount of compute they might be able to perform as well as AlphaChip? Or at least that the comparison would be fairer? Are the other methods scalable in that way?

Yes, they are. The other approaches usually look like simulated annealing, which has several hyperparameters that control how much computing is used and improve results with more compute usage.

See my comment above - the Nature authors already did this, and tried a huge hyperparameter sweep for SA, and RL still won.

See appendix of the Nature article: rdcu.be/cmedX

Re: Jeff Dean responds to EDA industry about AlphaChip

#156

Post by wholehog »

gabegobblegoldi wrote: post #135
Keyframe wrote: post #129

At this point in time, why wouldn't we give at least benefit of the doubt to Jeff Dean immediately? His track record is second to none, and he's still going strong. Has something happened that cast a shadow on him? Sometimes it is the messenger that brings in the weight.

Looks like he aligned himself with the wrong folks here. He is a system builder at heart but not an expert in chip design or EDA. And also not really an ML researcher. Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. His focus on this project didn’t help with his case at Google. They moved all the important stuff away from him and gave it to Demis last year and…

> Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out.

Was the TPU physical design team also taken in? And also MediaTek? And also TF-Agents, which publicly said they re-produced the AlphaChip method and results exactly?

Re: Jeff Dean responds to EDA industry about AlphaChip

#157

Post by wholehog »

dogleg77 wrote: post #143
alsodumb wrote: post #137

Earlier quoted context omitted.

I mean Jeff Dean is probably more ML researcher than probably 90% of the ML researchers out there. Sure, he may not be working on state of the art stuff himself; but he's too up the chain to do that.

That's an appeal to authority, and not an effective one. Jeff Dean doesn't have a good track record in chip design.

What are you even talking about? Jeff had a hand in TPU, which is so successful that all other AI companies are trying to clone this project and spin up their own efforts to make custom AI chips.

Re: Jeff Dean responds to EDA industry about AlphaChip

#158

Post by wholehog »

fxtentacle wrote: post #125
segmondy wrote: post #117

It's ridiculous how expensive the wrong hire can be https://www.wired.com/story/google-brain-ai-researcher-fired...

It seems that Chatterjee - the bad guy in your linked article - is now suing Google because he thinks he got canned for pointing out that his boss - Jeff Dean mentioned in the article discussed here - was knowingly publishing fraudulent claims. "To be clear, we do NOT have evidence to believe that RL outperforms academic state-of—art and strongest commercial macro placers. The comparisons for the latter were done so…

You're linking to his amended complaint - his original complaint was thrown out because it alleged things like "Google's motto is don't be evil, but they were evil, thus defrauding me."

According to a Google investigator's sworn statement, he admitted that he didn't have evidence to suspect the AlphaChip authors of fraud: "he stated that he suspected that the research being conducted by Goldie and Mirhoseini was fraudulent, but also stated that he did not have evidence to support his suspicion of fraud".

I feel like if someone persistently makes unsupported allegations of fraud, they should not be surprised if they get shown the door.

The comparison against commercial autoplacers might be this one (from That Chip Has Sailed - https://arxiv.org/pdf/2411.10053):

"In May of 2020, we performed a blind internal study[12] comparing our method against the latest version of two leading commercial autoplacers. Our method outperformed both, beating one 13 to 4 (with 3 ties) and the other 15 to 1 (with 4 ties). Unfortunately, standard licensing agreements with commercial vendors prohibit public comparison with their offerings."

[12] - "Our blind study compared RL to human experts and commercial autoplacers on 20 TPU blocks. First, the physical design engineer responsible for placing a given block ranked anonymized placements from each of the competing methods, evaluating purely on final QoR metrics with no knowledge of which method was used to generate each placement. Next, a panel of seven physical design experts reviewed each of the rankings and ties. The comparisons were unblinded only after completing both rounds of evaluation. The result was that the best placement was produced most often by RL, followed by human experts, followed by commercial autoplacers."

Re: Jeff Dean responds to EDA industry about AlphaChip

#159

Post by wholehog »

RicoElectrico wrote: post #95
_cs2017_ wrote: post #59

Curious why there's so much emotion and unpleasantness in this dispute? How did it evolve from the boring academic argument about benchmarks, significance, etc to a battle of personal attacks?

Making extraordinary claims without a way to replicate it. And then running to the press, which will swallow anything. Because "AI designs AI... umm... I mean chips" sounds futuristic to a liberal-arts majors (and apparently programmers too, which I'd expect to know better and question everything "AI") The whole publication process seems dishonest, starting from publishing in Nature (why not ISCCC or something simila…

> The whole publication process seems dishonest, starting from publishing in Nature (why not ISCCC or something similar?)

Why would you publish in ISCCC when you can get into Nature?

Post reply on HN