Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jeff Dean responds to EDA industry about AlphaChip

twitter.com

Post reply on HN
Print view View original

141–150 of 221 posts

Re: Jeff Dean responds to EDA industry about AlphaChip

#141

Post by woodson »

amelius wrote: post #127
rowanG077 wrote: post #78

I don't get. Why isn't the model open if it works? If it isn't this is just a fart in the wind. If it is the findings should be straightforward to replicate.

Yes, the community should force Nature to up its standards or ditch it. Software replication should be trivial in this day and age.

All these papers doing "research" on how to better prompt ChatGPT would be unpublishable then, given that API access to older models gets retired, so the findings of these papers can no longer be reproduced.

(I agree with you in principle; my example above is meant to show that standards for things such as reproducibility aren't easily defined. There are so many factors to consider.)

Re: Jeff Dean responds to EDA industry about AlphaChip

#142

Post by sdenton4 »

vighneshiyer wrote: post #122
Jabbles wrote: post #98

Earlier quoted context omitted.

You're saying that if the other methods were given the equivalent amount of compute they might be able to perform as well as AlphaChip? Or at least that the comparison would be fairer? Are the other methods scalable in that way?

Existing mixed-placement algorithms depend on hyperparameters, heuristics, and initial states / randomness. If afforded more compute resources, they can explore a much wider space and in theory come up with better solutions. Some algorithms like simulated annealing are easy to modify to exploit arbitrarily more compute resources. Indeed, I believe the comparison of AlphaChip to alternatives would be fairer if compute…

That sounds like future work for simulated annealing fans to engage in, quite honestly, rather than something that needs to be addressed immediately in a paper proposing an alternative method. The proposed method accomplished what it set out to do, surpassing current methods; others are free to explore different hyperparameters to surpass the quality again... This is, ultimately, why we build benchmark tasks: if you want to prove you know how to do it better, one is free to just go do it better instead of whining about what the competition did or didn't try on one's behalf.

Re: Jeff Dean responds to EDA industry about AlphaChip

#143

Post by dogleg77 »

alsodumb wrote: post #137
gabegobblegoldi wrote: post #135

Earlier quoted context omitted.

Looks like he aligned himself with the wrong folks here. He is a system builder at heart but not an expert in chip design or EDA. And also not really an ML researcher. Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. His focus on this project didn’t help with his case at Google. They moved all the important stuff away from him and gave it to Demis last year and…

I mean Jeff Dean is probably more ML researcher than probably 90% of the ML researchers out there. Sure, he may not be working on state of the art stuff himself; but he's too up the chain to do that.

That's an appeal to authority, and not an effective one. Jeff Dean doesn't have a good track record in chip design.

Re: Jeff Dean responds to EDA industry about AlphaChip

#144

Post by dogleg77 »

Keyframe wrote: post #129

At this point in time, why wouldn't we give at least benefit of the doubt to Jeff Dean immediately? His track record is second to none, and he's still going strong. Has something happened that cast a shadow on him? Sometimes it is the messenger that brings in the weight.

That made sense when Jeff Dean gave talks in 2020, 2021, and 2022. He is now responding to skepticism from the EDA community by unscholarly personal attacks and vague references to "many companies" using the work. He is beyond benefit of the doubt, and into the realm of probable cause.

Re: Jeff Dean responds to EDA industry about AlphaChip

#145

Post by t_serpico »

gabegobblegoldi wrote: post #135
Keyframe wrote: post #129

At this point in time, why wouldn't we give at least benefit of the doubt to Jeff Dean immediately? His track record is second to none, and he's still going strong. Has something happened that cast a shadow on him? Sometimes it is the messenger that brings in the weight.

Looks like he aligned himself with the wrong folks here. He is a system builder at heart but not an expert in chip design or EDA. And also not really an ML researcher. Some would say he got taken for a ride by a young charismatic grifter and is now in too deep to back out. His focus on this project didn’t help with his case at Google. They moved all the important stuff away from him and gave it to Demis last year and…

I don't think he got taken for a ride. Rather, he also wanted to believe that AlphaChip would be as revolutionary as it claimed to be and chose to ignore Chaterjee's reservations. Understandable, given all the AlphaX models coming out around that timeframe.

Re: Jeff Dean responds to EDA industry about AlphaChip

#146

Post by dogleg77 »

alsodumb wrote: post #139
choppaface wrote: post #47

Earlier quoted context omitted.

The result is minor AND Google spent a (relative) lot of money to achieve it (especially in the eyes of the new CFO). Jeff Dean is desperately trying to save the prestige of the research (in a very insular, Google-y way) because he wants to save the 2017-era economically-not-viable blue sky culture where Tensorflow & the TPU flourished and the transformer was born. But the reality is that Google’s core businesses are…

What makes you say TPU has zero chance against growing NVIDIA? If anything, now is the best time for TPU to grow and I'd say investing in TPU gave Google an edge. There is no other large scale LLM that was trained on anything but NVIDIA GPUs. Gemini is the only exception. Every big company is scrambling to make their own hardware in the AI era while Google already has it. Everyone I know who worked with TPUs loves ho…

Good point, but Google is buying Nvidia GPUs for some reason. Please remind me who's buying TPUs.

Re: Jeff Dean responds to EDA industry about AlphaChip

#147

Post by dogleg77 »

btilly wrote: post #49
bsder wrote: post #40

Earlier quoted context omitted.

> Why do a non-zero amount of people have seemingly religious beliefs about this topic on one side or the other? Because lots of engineers are being told by managers "Why aren't we using that tool?" and a bunch of engineers are stuck saying "Because it doesn't actually work." aka "Google is lying through their teeth." to which the response is "Oh, so you know better than Google?" to which the reponse is "Yeah, actual…

And do you believe that that is what's happening in this case? If you have personal experience with Jeff Dean et al that you're willing to share, I'd be interested in hearing about it. From where I'm sitting it looks like, "Google spent a fortune on deep learning, and got a small but real win. People who don't like Google failed to follow Google's recipe and got a large and easily replicated loss." It's not even clea…

When Google published the Nature article, Nature included a rosy intro article by a leading expert in chip design. His name was Andrew Kahng, and he apparently liked Google at the time. But when he dug into Google code (released way after publication), he retracted his intro and co-authored the Cheng et al article. You see how your theory breaks down here.

Re: Jeff Dean responds to EDA industry about AlphaChip

#148

Post by dogleg77 »

btilly wrote: post #72
bsder wrote: post #62

Earlier quoted context omitted.

> From where I'm sitting it looks like, "Google spent a fortune on deep learning, and got a small but real win. People who don't like Google failed to follow Google's recipe and got a large and easily replicated loss." From where I'm sitting it looks like Google cooked the books maximally, barely beat humans let alone state of the art algorithms, published a crappy article in Nature because it would never have passed…

Look, either the follow-up article did pretraining or not. Jeff Dean is claiming that the importance of pretraining was mentioned 37 times and the follow-up didn't do it. That sounds easy to verify. Likewise the importance of spending 20x as much money on the training portion seems easy to verify, and significant. That they would fail to properly test against industry standard workbenches seems reasonable to me. This…

Reading those papers and looking at the code, it doesn't look easy. However, let's imagine that the Cheng et al team comes back with results for pretraining a few months from now, and they support the conclusions of their earlier paper. What should they do to help everyone reach a conclusion?

Re: Jeff Dean responds to EDA industry about AlphaChip

#149

Post by mnau »

throwaway2037 wrote: post #61
quadrature wrote: post #45

Earlier quoted context omitted.

> How the hell would you verify an AI-generated silicon design? I think you're asking a different question, but in the context of the OP researchers are exploring AI for solving deterministic but intractable problems in the field of chip design and not generating designs end to end. Here's an excerpt from the paper. "The objective is to place a netlist graph of macros (e.g., SRAMs) and standard cells (logic gates, su…

> Despite decades of research on this problem, it is still necessary for human experts to iterate for weeks with the existing placement tools, in order to produce solutions that meet multi-faceted design criteria. Ironically, this sounds a lot like building a bot to play StarCraft, which is exactly what AlphaStar did. I had no idea that EDA layout is still so difficult and manual in 2024. This seems like a very worth…

This is not a board where you put resistors, capacitors and ICs on a substrate.

These are chip layouts used for fabbing chips. I don't think you will find many open source designs.

EDAs works closely with foundries (TSMC, Samsung, GlobalFoundaries). This is the bleeding edge stuff to get the best performance for NVIdia or AMD or Intel.

As an individual, it's very hard and expensive to fab your chip (though there are companies that pool multiple designs).

Re: Jeff Dean responds to EDA industry about AlphaChip

#150

Post by mnau »

lisper wrote: post #43
AtlasBarfed wrote: post #41

How the hell would you verify an AI-generated silicon design? Like, for a CPU, you want to be sure it behaves properly for the given inputs. Anyone remember that floating point error in, was it Pentium IIs or Pentium IIIs? I mean, I guess if the chip is designed for AI, and AIs are inherently nonguaranteed output/responses, then the AI chip design being nonguaranteed isn't any difference in nonguarantees. Unless it i…

> How the hell would you verify an AI-generated silicon design? The same way you verify a human-generated one. > Anyone remember that floating point error in, was it Pentium IIs or Pentium IIIs? That was 1994. The industry has come a long way in the intervening 30 years.

> The industry has come a long way in the intervening 30 years.

Didn't Intel screw up design recently? Oxidation and degradation of ring bus IIRC.

Post reply on HN