Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YC is wrong about LLMs for chip design

zach.be

Post reply on HN
Print view View original

191–200 of 283 posts

Re: YC is wrong about LLMs for chip design

#191

Post by tim333 »

kuhewa wrote: post #126
aubanel wrote: post #110

I know nothing about chip design. But saying "Applying AI to field X won't work, because X is complex, and LLMs currently have subhuman performance at this" always sounds dubious. VCs are not investing in the current LLM-based systems to improve X, they're investing in a future where LLM based systems will be 100x more performant. Writing is complex, LLMs once had subhuman performance, and yet. Digital art. Music (se…

> Writing is complex, LLMs once had subhuman performance, And now they can easily replace mediocre human performance, and since they are tuned to provide answers that appeal to humans that is especially true for these subjective value use cases. Chip design doesn't seem very similar. Seems like a case where specifically trained tools would be of assistance. For some things, as much as generalist LLMs have surprised a…

When people say LLM I think they are often thinking of neural network approaches in general rather than just text based even if the letters do stand for language model. And there's overlap eg. Gemini does language but is multi modal. If you skip that you get things like AlphaZero which did beat Stockfish https://en.wikipedia.org/wiki/AlphaZero

Re: YC is wrong about LLMs for chip design

#192

Post by tim333 »

>If Gary Tan and YC believe that LLMs will be able to design chips 100x better than humans currently can, they’re significantly underestimating the difficulty of chip design, and the expertise of chip designers.

I don't think he's arguing that. More that ASICs can be 100x better than CPUs for say crypto mining and that using LLM type stuff it may be possible to make them for other applications where there is less money available to hire engineers.

(the YC request https://www.ycombinator.com/rfs-build#llms-for-chip-design)

Re: YC is wrong about LLMs for chip design

#193

Post by rm445 »

lolinder wrote: post #13

One of the consistent problems I'm seeing over and over again with LLMs is people forgetting that they're limited by the training data. Software engineers get hyped when they see the progress in AI coding and immediately begin to extrapolate to other fields—if Copilot can reduce the burden of coding so much, think of all the money we can make selling a similar product to XYZ industries! The problem with this extrapol…

Many other industries haven't yet been fully eaten by software. All kinds of data is locked away and in proprietary formats, and is generated by humans without much automation. I don't think we know where exactly the frontiers are, once someone puts in the work to build large datasets, and automates creation of synthetic training data. Whole industries could suddenly flip from 'impossible' to 'easy' for AI.

Re: YC is wrong about LLMs for chip design

#194

Post by necovek »

paulsutter wrote: post #156
necovek wrote: post #121

Earlier quoted context omitted.

> Ever try to drive a nail without a hammer? I've driven a nail with a rock, a pair of pliers, a wrench, even with a concrete wall and who knows what else! I didn't need to be told if these can be used to drive a nail, and I looked at things available, looked for a flat surface on them and good grip, considered their hardness, and then simply used them. So if we only give them the "right" tools, they'll remain very l…

Those are all tools !! Congratulations If you’re creative enough to figure out different tools for humans, you are creative enough to figure out different tools for LLMs

No disagreement there, but if we've got the tools, do we really need an LLM to drive them (it still requires building an adapter from LLM to those tools)?

What is the added value of that combo and at what cost?

Re: YC is wrong about LLMs for chip design

#195

Post by mikewarot »

I agree LLMs aren't ready to design ASICs. It's likely that in a decade or less, they'll be ready for the times you absolutely need to squeeze out every square nanometer, picosecond, femtojoule, or nanowatt.

Gary Tan's was right[1] in that there is a fundamental inefficiency inherent in the von Neumann architecture we're all using. This gross impedance mismatch[4] is a great opportunity for innovation.

Once ENIAC was "improved" from its original structure to a general purpose compute device in the von Neumann style, it suffered a 83% loss in performance[2] Everything since is 80 years of premature optimization that we need to unwind. It's the ultimate pile of technical debt.

Instead of throwing maximum effort into making specific workloads faster, why not build a chip that can make all workloads faster instead, and let economy of scale work for everyone?

I propose (and have for a while[3]) a general purpose solution.

A systolic array of simple 4 bits in, 4 bits out, Look Up Tables (LUTs) latched so that timing issues are eliminated, could greatly accelerate computation, in a far nearer timeframe.

The challenges are that it's a greenfield environment, with no compilers (though it's probable that LLVM could target it), and a bus number of 1.

[1] https://www.ycombinator.com/rfs-build#llms-for-chip-design

[2] https://en.wikipedia.org/wiki/ENIAC#Improvements

[3] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

[4] https://en.wikipedia.org/wiki/Impedance_matching

Re: YC is wrong about LLMs for chip design

#196

Post by brookst »

beepbooptheory wrote: post #144
brookst wrote: post #92

Earlier quoted context omitted.

Sure. I mean, humans are very good at building businesses and technologies that are resilient to human fallibility. So when we think of applications where LLMs might replace or augment humans, it’s unsurprising that their fallible nature isn’t a showstopper. Sure, EDA tools are deterministic, but the humans who apply them are not. Introducing LLMs to these processes is not some radical and scary departure, it’s an it…

Ok yeah. I think the thing that trips me up with this argument then is just, yes, when you regard humans in a certain neuroscientific frame and consider things like consciousness or language or will, they are fundamentally nondeterministic. But that isn't the frame of mind of the human engineer who does the work or even validates it. When the engineer is working, they aren't seeing themselves as some black box which…

We start from similar places but get to very different conclusions.

I accept that I’m fallible, both in my areas of expertise and in all the meta stuff around it. I code bugs. I omit requirements. Not often, and there are mental and technical means to minimize, but my work, my org’s structure, my company’s processes are all designed to mitigate human fallibility.

I’m not interested in “defending” AI models. I’m just saying that their weaknesses are qualitatively similar to human weaknesses, and as such, we are already prepared to deal with those weaknesses as long as we are aware of them, and as long as we don’t make the mistake of thinking that because they use transistors they should be treated like a mostly deterministic piece of software where one unit test pass means it is good.

I think you’re reading some kind of value judgement on consciousness into what is really just a pragmatic approach to slotting powerful but imperfect agents into complex systems. It seems obvious to me, and without any implications as to human agency.

Re: YC is wrong about LLMs for chip design

#197

Post by zachglabman »

LLMs are wrong for most things imo. LLMs are great conversational assistants, but there is very little linguistic rigor to them, if any. They have almost no generalization ability, and anecdotally they fall for the same syntactic pitfalls they've fallen for since BERT. Models have gotten so good at predicting this n-dimensional "function" that sounds like human speech, we're getting distracted from seeing their actual purpose and trying to apply them to all sorts of problems that rely on more than text-based training data.

Language is cool and immensely useful. LLMs, however, are fundamentally flawed from their basic assumptions about how language works. The distribution hypothesis is good for paraphrasing and summarization, but pretty atrocious for real reasoning. The concept of an idea living in a semantic "space" is incompatible with simple vector spaces, and we are starting to see this actually matter in minutia with scaling laws coming into play. Chip design is a great example of where we cannot rely on language alone to solve all our problems.

I hope to be proven wrong, but still not sold on AGI being within reach. We'll probably need some pretty significant advancements in large quantitative models, multi-modal models and smaller, composable models of all types before we see AGI

Re: YC is wrong about LLMs for chip design

#198

Post by pkkim »

Glad to see that the author is highlighting verification as the important factor in design productivity.

We at Silogy [0] are directly targeting the problem of verification productivity using AI agents for test debugging. We analyze code (RTL, testbench, specs, etc.) along with logs and waveforms, and incorporate interactive feedback from the engineer as needed to refine the hypothesis.

[0] https://silogy.io/

Re: YC is wrong about LLMs for chip design

#199

Post by shash »

aubanel wrote: post #110

I know nothing about chip design. But saying "Applying AI to field X won't work, because X is complex, and LLMs currently have subhuman performance at this" always sounds dubious. VCs are not investing in the current LLM-based systems to improve X, they're investing in a future where LLM based systems will be 100x more performant. Writing is complex, LLMs once had subhuman performance, and yet. Digital art. Music (se…

In this specific case, it's hard to see how LLMs can get you from here to there. the problem isn't the boilerplate code like when you build a react website with them, but the really novel architectures (and architectural decisions, more importantly) that you need to make along the way. Some of those can seem very arbitrary and require deep understanding to pull off. You can't just use language-like tokens to reason this out. Fundamental understanding of the laws and rules of thumb are important.

Re: YC is wrong about LLMs for chip design

#200

Post by duped »

jjk166 wrote: post #177
duped wrote: post #167

Earlier quoted context omitted.

AI still has subhuman performance for art. It feels like the venn diagram of people who are bullish on LLMs and people who don't understand logistic curves is a circle.

You ask 100,000 humans each to make a photo realistic rendering of a alpaca playing basketball on the moon in 90 seconds, an LLM is going to outperform every single one of them.

That's not a meaningful benchmark for valuing art or creating art
Post reply on HN