Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YC is wrong about LLMs for chip design

zach.be

Post reply on HN
Print view View original

121–130 of 283 posts

Re: YC is wrong about LLMs for chip design

#121

Post by necovek »

paulsutter wrote: post #17

Generative models are bimodal - in certain tasks they are crazy terrible , and in certain tasks they are better than humans. The key is to recognize which is which. And much more important: - LLMs can suddenly become more competent when you give them the right tools, just like humans. Ever try to drive a nail without a hammer? - Models with spatial and physical awareness are coming and will dramatically broaden what’…

> Ever try to drive a nail without a hammer?

I've driven a nail with a rock, a pair of pliers, a wrench, even with a concrete wall and who knows what else!

I didn't need to be told if these can be used to drive a nail, and I looked at things available, looked for a flat surface on them and good grip, considered their hardness, and then simply used them.

So if we only give them the "right" tools, they'll remain very limited by us not thinking about possible jobs they'll appear as if they know how to do and they don't.

The problem is exactly that: they "pretend" to know how to drive a nail but not really.

Re: YC is wrong about LLMs for chip design

#122

Post by jrflowers »

aubanel wrote: post #110

I know nothing about chip design. But saying "Applying AI to field X won't work, because X is complex, and LLMs currently have subhuman performance at this" always sounds dubious. VCs are not investing in the current LLM-based systems to improve X, they're investing in a future where LLM based systems will be 100x more performant. Writing is complex, LLMs once had subhuman performance, and yet. Digital art. Music (se…

I like this reasoning. It is shortsighted to say that LLMs aren’t well-suited to something (because we cannot tell the future) but it is not shortsighted to say that LLMs are well-suited to something (because we cannot tell the future)

Re: YC is wrong about LLMs for chip design

#123

Post by jaxr »

I don't know the space well enough, but I think the missing piece is that YC 's investment horizon is typically 10+ years. Not only LLMs could get massively better, but the chip industry could be massively disrupted with the right incentives. My guess is that that is YC's thesis behind the ask.

Re: YC is wrong about LLMs for chip design

#124

Post by zachbee »

mgraczyk wrote: post #108

I worked on the Qualcomm DSP architecture team for a year, so I have a little experience with this area but not a ton. The author here is missing a few important things about chip design. Most of the time spent and work done is not writing high performance Verilog. Designers spent a huge amount of time answering questions, writing documentation, copying around boiler plate, reading obscure manuals and diagrams, etc.…

Author here -- I don't disagree! I actually noted this in the article:

> Well, it turns out that LLMs are also pretty valuable when it comes to chips for lucrative markets -- but they won’t be doing most of the design work. LLM copilots for Verilog are, at best, mediocre. But leveraging an LLM to write small snippets of simple code can still save engineers time, and ultimately save their employers money.

I think designers getting 2x faster is probably optimistic, but I also could be wrong about that! Most of my chip design experience has been at smaller companies, with good documentation, where I've been focused on datapath architecture & design, so maybe I'm underestimating how much boilerplate the average engineer deals with.

Regardless, I don't think LLMs will be designing high-performance datapath or networking Verilog anytime soon.

Re: YC is wrong about LLMs for chip design

#125

Post by cwzwarich »

marcosdumay wrote: post #16
fsndz wrote: post #9

They want to throw LLMs at everything even if it does not make sense. Same is true for all the AI agent craze: https://medium.com/thoughts-on-machine-learning/langchains-s...

If feels like the entire world has gone crazy. Even the serious idea that the article thinks could work is throwing the unreliable LLMs at verification ! If there's any place you can use something that doesn't work most of the time, I guess it's there.

If your LLM is producing a proof that can be checked by another program, then there’s nothing wrong with their reliability. It’s just like playing a game whose rules are a logical system.

Re: YC is wrong about LLMs for chip design

#126

Post by kuhewa »

aubanel wrote: post #110

I know nothing about chip design. But saying "Applying AI to field X won't work, because X is complex, and LLMs currently have subhuman performance at this" always sounds dubious. VCs are not investing in the current LLM-based systems to improve X, they're investing in a future where LLM based systems will be 100x more performant. Writing is complex, LLMs once had subhuman performance, and yet. Digital art. Music (se…

> Writing is complex, LLMs once had subhuman performance,

And now they can easily replace mediocre human performance, and since they are tuned to provide answers that appeal to humans that is especially true for these subjective value use cases. Chip design doesn't seem very similar. Seems like a case where specifically trained tools would be of assistance. For some things, as much as generalist LLMs have surprised at skill in specific tasks, it is very hard to see how training on a broad corpus of text could outperform specific tools — for first paragraph do you really think it is not dubious to think a model trained on text would outperform Stockfish at chess?

Re: YC is wrong about LLMs for chip design

#127

Post by OkGoDoIt »

klabb3 wrote: post #68
brookst wrote: post #24

Earlier quoted context omitted.

You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems?

Yes I do! Is that some sort of gotcha? If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years”, I’m going to pick the script. Who wouldn’t? Until machines can reliably understand, operate and self-correct independently, I’d rather not give up debuggability and understandability.

I think this comment and the parent comment are talking about two different things. One of you is talking about using nondeterministic ML to implement the actual core logic (an automated script or asking Dave to do it manually), and one of you is talking about using it to design the logic (the equivalent of which is writing that automated script).

LLM’s are not good at actually doing the processing, they are not good at math or even text processing at a character level. They often get logic wrong. But they are pretty good at looking at patterns and finding creative solutions to new inputs (or at least what can appear creative, even if philosophically it’s more pattern matching than creativity). So an LLM would potentially be good at writing a first draft of that script, which Dave could then proofread/edit, and which a standard deterministic computer could just run verbatim to actually do the processing. Eventually maybe even Dave’s proofreading would be superfluous.

Tying this back to the original article, I don’t think anyone is proposing having an LLM inside a chip that processes incoming data in a non-deterministic way. The article is about using AI to design the chips in the first place. But the chips would still be deterministic, the equivalent of the script in this analogy. There are plenty of arguments to make about LLM‘s not being good enough for that, not being able to follow the logic or optimize it, or come up with novel architectures. But the shape of chip design/Verilog feels like something that with enough effort, an AI could likely be built that would be pretty good at it. All of the knowledge that those smart knowledgeable engineers which are good at writing Verilog have built up can almost certainly be represented in some AI form, and I wouldn’t bet against AI getting to a point where it can be helpful similarly to how Copilot currently is with code completion. Maybe not perfect anytime soon, but good enough that we could eventually see a path to 100%. It doesn’t feel like there’s a fundamental reason this is impossible on a long enough time scale.

Re: YC is wrong about LLMs for chip design

#130

Post by kuhewa »

famouswaffles wrote: post #94
klabb3 wrote: post #68

Earlier quoted context omitted.

Yes I do! Is that some sort of gotcha? If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years”, I’m going to pick the script. Who wouldn’t? Until machines can reliably understand, operate and self-correct independently, I’d rather not give up debuggability and understandability.

>If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years” If you could that would be nice wouldn't it? And if you couldn't? If people were saying, "let's replace Casio Calculators with interfaces to GPT" then that would be crazy and I would wholly agree with you but by and large, the processes people are scrambling to place LLMs in are ones…

> by and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail

I'm pretty sure they are scrambling to put them absolutely anywhere it might save or make a buck (or convince an investor that it could)

Post reply on HN