When I think of AI in chip design, optimizations like these come to mind,
https://optics.ansys.com/hc/en-us/articles/360042305274-Inve...
https://optics.ansys.com/hc/en-us/articles/33690448941587-In...
YC is wrong about LLMs for chip design
71–80 of 283 posts
Re: YC is wrong about LLMs for chip design
#72Earlier quoted context omitted.
yes thats how we progress this is how the internet boom happened as well everything became . com then the real workable businesses were left and all the unworkable things were gone. Recently I came across some one advertising an LLM to generate fashion magazine shoot in Pakistan at 20-25% of the cost. It hit me then that they are undercutting the fashion shoot of country like Pakistan which is already cheaper by 90-9…
The annoying part, a lot of money could be funneled into these unworkable businesses in the process, crypto being a good example. And these unworkable businesses tend to try to continue getting their way into the money somehow regardless. Most recent example was funneling money from Russia into Trump’s campaign.
> The annoying part, a lot of money could be funneled into these unworkable businesses in the process, crypto being a good example
There was a thread here about why ycombinator invests into several competing startups. The answer is success is often more about connections and politics than the product itself. And crypto, yes, is a good example of this. Musk will get his $1B in bitcoins back for sure.
> Most recent example was funneling money from Russia into Trump’s campaign.
Musk again?
Re: YC is wrong about LLMs for chip design
#73Had to nop out at "just next token prediction". This article isn't worth your time.
Re: YC is wrong about LLMs for chip design
#74The article seems to be be based on the current limitations of LLMs. I don't think YC and other VCs are betting on what LLMs can do today, I think they are betting on what they might be able to do in the future. As we've seen in the recent past, it's difficult to predict what the possibilities are for LLMS and what limitations will hold. Currently it seems pure scaling won't be enough, but I don't think we've reached…
Tomorrow, LLMs will be able to perform slightly below-average versions of whatever humans are capable of doing tomorrow. Because they work by predicting what a human would produce based on training data.
It's worth considering
1) all the domains there is no training data
Many professions are far less digital than software, protect IP more, and are much more akin to an apprenticeship system.
2) the adaptability of humans in learning vs any AI
Think about how many years we have been trying to train cars to drive, but humans do it with a 50 hours training course.
3) humans ability to innovate vs AIs ability to replicate
A lot of creative work is adaptation, but humans do far more than that in synthesizing different ideas to create completely new works. Could an LLM produce the 37th Marvel movie? Yes probably. Could an LLM create.. Inception? Probably not.
Re: YC is wrong about LLMs for chip design
#75> If Gary Tan and YC believe that LLMs will be able to design chips 100x better than humans currently can, they’re significantly underestimating the difficulty of chip design, and the expertise of chip designers. While LLMs are capable of writing functional Verilog sometimes, their performance is still subhuman. [...] LLMs primarily pump out mediocre Verilog code.
What is the quality of Verilog code output by humans? Is it good enough so that a complex AI chip can be created? Or does the human need to use tools in order to generate this code?
I've got the feeling that LLMs will be capable of doing everything a human can do, in terms of thinking. There shouldn't be an expectation that an LLM is able to do everything, which in this context would be thinking about the chip and creating the final files in a single pass and without external help. And with external help I don't mean us humans, but tools which are specialized and also generate some additional data (like embeddings) which the LLM (or another LLM) can use in the next pass to evaluate the design. And if we humans have spent enough time in creating these additional tools, there will come a time when LLMs will also be able to create improved versions of them.
I mean, when I once randomly checked the content of a file in The Pile, I found an Craigslist "ad" for an escort offering her services. No chip-generating AI does need to have this in its parameters in order to do its job. So there is a lot of room for improvement and this improvement will come over time. Such an LLM doesn't need to know that much about humans.
Re: YC is wrong about LLMs for chip design
#76I don’t mind LLMs in the ideation and learning phases, which aren’t reproducible anyway. But I still find it hard to believe engineers of all people are eager to put a slow, expensive, non-deterministic black box right at the core of extremely complex systems that need to be reliable, inspectable, understandable…
You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems?
In a reductive sense, this passage might as well read "You find it hard to believe that entropy is the source of other entropic reactions?"
No, I'm just disappointed in the decision of Black Box A and am bound to be even more disappointed by Black Box B. If we continue removing thoughtful design from our systems because thoughtlessness is the default, nobody's life will improve.
Re: YC is wrong about LLMs for chip design
#77I don’t mind LLMs in the ideation and learning phases, which aren’t reproducible anyway. But I still find it hard to believe engineers of all people are eager to put a slow, expensive, non-deterministic black box right at the core of extremely complex systems that need to be reliable, inspectable, understandable…
100% agree. While I can’t find all the sources right now, [1] and its references could be a good starting point for further exploration. I recall there being a proof or conjecture suggesting that it’s impossible to build an "LLM firewall" capable of protecting against all possible prompts—though my memory might be failing me
Re: YC is wrong about LLMs for chip design
#78Earlier quoted context omitted.
>The article seems to be be based on the current limitations of LLMs. I don't think YC and other VCs are betting on what LLMs can do today, I think they are betting on what they might be able to do in the future. Do we know what LLMs will be able to do in the future? And even if we know, the startups have to work with what they have now, until that future comes. The article states that there's not much to work with.
Show me a successful startup that was predicated on the tech they’re working with not advancing?
Every single software service that has ever provided an Android or iOS application, for starters.
Re: YC is wrong about LLMs for chip design
#79Earlier quoted context omitted.
If feels like the entire world has gone crazy. Even the serious idea that the article thinks could work is throwing the unreliable LLMs at verification ! If there's any place you can use something that doesn't work most of the time, I guess it's there.
It's similar in regular programming - LLMs are better at writing test code than actual code. Mostly because it's simpler (P vs NP etc), but I think also because it's less obvious when test code doesn't work. Replace all asserts with expected ==expected and most people won't notice.
> Replace all asserts with expected == expected and most people won't notice.
It’s too resource intensive for all code, but mutation testing is pretty good at finding these sorts of tests that never fail. https://pitest.org/
Re: YC is wrong about LLMs for chip design
#80LLMs have a long way to go in the world of EDA. A few months ago I saw a post on LinkedIn where someone fed the leading LLMs a counter-intuitively drawn circuit with 3 capacitors in parallel and asked what the total capacitance was. Not a single one got it correct - not only did they say the caps were in series (they were not) it even got the series capacitance calculations wrong. I couldn’t believe they whiffed it a…
[deleted]