Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YC is wrong about LLMs for chip design

zach.be

Post reply on HN
Print view View original

211–220 of 283 posts

Re: YC is wrong about LLMs for chip design

#211

Post by hulitu »

OkGoDoIt wrote: post #127
klabb3 wrote: post #68

Earlier quoted context omitted.

Yes I do! Is that some sort of gotcha? If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years”, I’m going to pick the script. Who wouldn’t? Until machines can reliably understand, operate and self-correct independently, I’d rather not give up debuggability and understandability.

I think this comment and the parent comment are talking about two different things. One of you is talking about using nondeterministic ML to implement the actual core logic (an automated script or asking Dave to do it manually), and one of you is talking about using it to design the logic (the equivalent of which is writing that automated script). LLM’s are not good at actually doing the processing, they are not good…

> So an LLM would potentially be good at writing a first draft of that script, which Dave could then proofread/edit, and which a standard deterministic computer could just run verbatim to actually do the processing

Or Dave could write a first draft of that script, saving him the time needed to translate what the LLM composed.

Re: YC is wrong about LLMs for chip design

#213

Post by ein0p »

klabb3 wrote: post #10

I don’t mind LLMs in the ideation and learning phases, which aren’t reproducible anyway. But I still find it hard to believe engineers of all people are eager to put a slow, expensive, non-deterministic black box right at the core of extremely complex systems that need to be reliable, inspectable, understandable…

LLMs can be fully deterministic BTW, depending on the sampling method used. Some methods do not have a random component. As to the rest, yeah - they aren't inspectable or understandable yet.

Re: YC is wrong about LLMs for chip design

#215

Post by rcxdude »

qwertox wrote: post #75

> If Gary Tan and YC believe that LLMs will be able to design chips 100x better than humans currently can, they’re significantly underestimating the difficulty of chip design, and the expertise of chip designers. While LLMs are capable of writing functional Verilog sometimes, their performance is still subhuman. [...] LLMs primarily pump out mediocre Verilog code. What is the quality of Verilog code output by humans?…

LLMs only reach the performance they do because of the sheer scale of data they ingest. Training them on less data doesn't work as well, or at least you will overfit like crazy on anything the size of current models. So the question is where are you going to get anywhere near the volume of verilog code as is present in The Pile? The total amount of verilog ever written is almost certainly a few orders of magnitude less.

Re: YC is wrong about LLMs for chip design

#216

Post by sincerely »

astrange wrote: post #189
jjk166 wrote: post #177

Earlier quoted context omitted.

You ask 100,000 humans each to make a photo realistic rendering of a alpaca playing basketball on the moon in 90 seconds, an LLM is going to outperform every single one of them.

Diffusion models aren't actually LLMs, they're a different architecture. Which makes it even weirder we invented them at the same time. Also, they might not be able to do it. eg most models can't generate "horse riding an astronaut" or "upside-down car".

To be fair, most humans can't draw any better than stick figures.

Re: YC is wrong about LLMs for chip design

#217

Post by bitwize »

mportela wrote: post #158
crabmusket wrote: post #150

Earlier quoted context omitted.

I studied mechatronics and did reasonably well... but in any electrical class I would just scrape by. I loved it but was apparently not suited to it. I remember a whole unit basically about transistors. On the software/mtrx side we were so happy treating MOSFETs as digital. Having to analyse them in more depth did my head in.

I had a similar experience, except Mechanical Engineering being my weakest area. Computer Science felt like a children's game compared to fluid dynamics...

Maybe this is why Sussman decided to approach understanding physics by way of programming.

Re: YC is wrong about LLMs for chip design

#218

Post by mikewarot »

alain94040 wrote: post #4

I agree with most of the technical points of the article. But there may still be value in YC calling for innovation in that space. The article is correctly showing that there is no easy win in applying LLMs to chip design. Either the market for a given application is too small, then LLMs can help but who cares, or the chip is too important, in which case you'd rather use the best engineers. Unlike software, we're not…

There is an obvious path forward, but apparently this is a minority opinion, possibly fringe. It doesn't make the traditional tradeoffs.

A bit level (non von Neumann) general purpose systolic array could greatly speed up AI computations, along with almost everything else. It's a chip to do general purpose computation.

The chip design is almost trivial. I'd expect someone with a few years of experience could knock it out in a few days. I hope to field a design in the next TinyTapeout (I'm on a fixed income, so I've had to wait a while)

The real problem is programming. We're talking vast greenfields that go on forever. There's no good way to target the architecture, you certainly wouldn't want to use Verilog or any other HDL.

Re: YC is wrong about LLMs for chip design

#219

Post by bsder »

mportela wrote: post #158
crabmusket wrote: post #150

Earlier quoted context omitted.

I studied mechatronics and did reasonably well... but in any electrical class I would just scrape by. I loved it but was apparently not suited to it. I remember a whole unit basically about transistors. On the software/mtrx side we were so happy treating MOSFETs as digital. Having to analyse them in more depth did my head in.

I had a similar experience, except Mechanical Engineering being my weakest area. Computer Science felt like a children's game compared to fluid dynamics...

They call it Thermogoddamics for a reason ...

Re: YC is wrong about LLMs for chip design

#220

Post by zahlman »

spamizbad wrote: post #21

LLMs have a long way to go in the world of EDA. A few months ago I saw a post on LinkedIn where someone fed the leading LLMs a counter-intuitively drawn circuit with 3 capacitors in parallel and asked what the total capacitance was. Not a single one got it correct - not only did they say the caps were in series (they were not) it even got the series capacitance calculations wrong. I couldn’t believe they whiffed it a…

I would expect an LLM's internal modeling to be on approximately the level of "this is a diagram of a capacitor circuit for some student's homework; electrical component calculations for homework tend to use the adding-in-reciprocal rule, because simple addition would be too straightforward for homework".

> “they didn’t even teach me this in 4 years of Electrical Engineering!” (Really? They don’t teach how capacitors block DC and how RC filters work????)

My experience with being an adult, in general, is that many people who went to university don't believe that any given course taught them anything meaningful.

I can absolutely believe that such people didn't learn and remember anything meaningful from those courses. Whether the course is to blame, is far more questionable.

Post reply on HN