Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YC is wrong about LLMs for chip design

zach.be

Post reply on HN
Print view View original

91–100 of 283 posts

Re: YC is wrong about LLMs for chip design

#91

Post by brookst »

jeltz wrote: post #65
brookst wrote: post #28

Earlier quoted context omitted.

Show me a successful startup that was predicated on the tech they’re working with not advancing?

Most? I can list tens of them easily. For example what advancements were required for Slack to be successful? Or Spotify (they got more successful due to smartphones and cheaper bandwidth but the business was solid before that)? Or Shopify?

Slack bet on ubiquitous, continuous internet access. Spotify bet on bandwidth costs falling to effectively zero. Shopify bet on D2C rising because improved search engines, increased internet shopping (itself a result of several tech trends plus demographic changes).

For a counterexample I think I’d look to non-tech companies. OrangeTheory maybe?

Re: YC is wrong about LLMs for chip design

#92

Post by brookst »

beepbooptheory wrote: post #33
brookst wrote: post #24

Earlier quoted context omitted.

You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems?

Can you actually like follow through with this line? I know there are literally tens of thousands of comments just like this at this point, but if you have chance, could you explain what you think this means? What should we take from it? Just unpack it a little bit for us.

Sure. I mean, humans are very good at building businesses and technologies that are resilient to human fallibility. So when we think of applications where LLMs might replace or augment humans, it’s unsurprising that their fallible nature isn’t a showstopper.

Sure, EDA tools are deterministic, but the humans who apply them are not. Introducing LLMs to these processes is not some radical and scary departure, it’s an iterative evolution.

Re: YC is wrong about LLMs for chip design

#93

Post by visarga »

sdesol wrote: post #70
croshan wrote: post #50

Earlier quoted context omitted.

An interpretation that makes sense to me: humans are non-deterministic black boxes already at the core of complex systems. So in that sense, replacing a human with AI is not unreasonable. I’d disagree, though: humans are still easier to predict and understand (and trust) than AI, typically.

With humans we have a decent understanding of what they are capable of. I trust a medical professional to provide me with medical advice and an engineer to provide me with engineering advice. With LLM, it can be unpredictable at times, and they can make errors in ways that you would not imagine. Take the following examples from my tool, which shows how GPT-4o and Claude 3.5 Sonnet can screw up. In this example, GPT-4…

It's actually a great topic - both humans and LLMs are black boxes. And both rely on patterns and abstractions that are leaky. And in the end it's a matter of trust, like going to the doctor.

But we have had extensive experience with humans, it is normal to have better defined trust, LLMs will be better understood as well. There is no central understander or truth, that is the interesting part, it's a "Blind men and the elephant" situation.

Re: YC is wrong about LLMs for chip design

#94

Post by famouswaffles »

klabb3 wrote: post #68
brookst wrote: post #24

Earlier quoted context omitted.

You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems?

Yes I do! Is that some sort of gotcha? If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years”, I’m going to pick the script. Who wouldn’t? Until machines can reliably understand, operate and self-correct independently, I’d rather not give up debuggability and understandability.

>If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years”

If you could that would be nice wouldn't it? And if you couldn't?

If people were saying, "let's replace Casio Calculators with interfaces to GPT" then that would be crazy and I would wholly agree with you but by and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail and humans excel or do decently (and that LLMs are making some headway in).

You're making the wrong distinction here. It's not Dave vs your nifty script. It's Dave or nothing at all.

There's no point comparing LLM performance to some hypothetical perfect understanding machine that doesn't exist.

You compare to the things its meant to replace - humans. How well can the LLM do this compared to Dave ?

Re: YC is wrong about LLMs for chip design

#95

Post by famouswaffles »

beepbooptheory wrote: post #33
brookst wrote: post #24

Earlier quoted context omitted.

You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems?

Can you actually like follow through with this line? I know there are literally tens of thousands of comments just like this at this point, but if you have chance, could you explain what you think this means? What should we take from it? Just unpack it a little bit for us.

Because people are not saying "let's replace Casio Calculators with interfaces to GPT!"

By and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail and humans excel or do decently (and that LLMs are making some headway in).

There's no point comparing LLM performance to some hypothetical perfect understanding machine that doesn't exist. It's nonsensical actually. You compare it to the performance of the beings it's meant to replace or augment - humans.

Replacing non-deterministic black boxes with potentially better performing non-deterministic black boxes is not some crazy idea.

Re: YC is wrong about LLMs for chip design

#96

Post by spencerchubb »

fsndz wrote: post #9

They want to throw LLMs at everything even if it does not make sense. Same is true for all the AI agent craze: https://medium.com/thoughts-on-machine-learning/langchains-s...

LLMs have powered products used by hundreds of millions, maybe billions. Most experiments will fail and that's okay, arguably even a good thing. Only time will tell which ones succeed

Re: YC is wrong about LLMs for chip design

#97

Post by namaria »

samatman wrote: post #63
layer8 wrote: post #39

Earlier quoted context omitted.

You could replace “LLM” in your comment with lots of other technologies. Why bet on LLMs in particular to escape their limitations in the near term?

Because YCombinator is all about r-selecting startup ideas, and making it back on a few of them generating totally outsized upside. I think that LLMs are plateauing, but I'm less confident that this necessarily means the capabilities we're using LLMs for right now will also plateau. That is to say it's distinctly possible that all the talent and money sloshing around right now will line up a new breakthrough architec…

> But if I had $100 million, and could bet $200 thousand that someone can make me billions on machine learning chip design or whatever, I'd probably entertain that bet. It's a numbers game.

Problem with this reasoning is twofold: start-ups will overfit to getting your money instead of creating real advances; competition amongst them will drive up the investment costs. Pretty much what has been happening.

Re: YC is wrong about LLMs for chip design

#98

Post by spencerchubb »

wslh wrote: post #40
fsndz wrote: post #9

They want to throw LLMs at everything even if it does not make sense. Same is true for all the AI agent craze: https://medium.com/thoughts-on-machine-learning/langchains-s...

This makes complete sense from an investor’s perspective, as it increases the chances of a successful exit. While we focus on the technical merits or critique here on HN/YC, investors are playing a completely different game. To be a bit acerbic, and inspired by Arthur C. Clarke, I might say: "Any sufficiently complex business could be indistinguishable from Theranos".

Theranos was not a "complex business". It was deliberate fraud and deception, and investors that were just gullible. The investors should have demanded to see concrete results

Re: YC is wrong about LLMs for chip design

#99

Post by MrsPeaches »

spamizbad wrote: post #21

LLMs have a long way to go in the world of EDA. A few months ago I saw a post on LinkedIn where someone fed the leading LLMs a counter-intuitively drawn circuit with 3 capacitors in parallel and asked what the total capacitance was. Not a single one got it correct - not only did they say the caps were in series (they were not) it even got the series capacitance calculations wrong. I couldn’t believe they whiffed it a…

How many words in the art of electronics? Could you give that as context and see if might help?

Re: YC is wrong about LLMs for chip design

#100

Post by namaria »

herval wrote: post #19
kokanee wrote: post #15

Earlier quoted context omitted.

Tomorrow, LLMs will be able to perform slightly below-average versions of whatever humans are capable of doing tomorrow. Because they work by predicting what a human would produce based on training data.

This severely discounts the fact that you’re comparing a model that _knows the average about everything_ to a single human’s capabilit. Also they can do it instantly, instead of having to coordinate many humans over long periods of time. You can’t straight up compare one LLM to one human

"Knows the average relationship amongst all words in the training data" ftfy
Post reply on HN