[dead]
Testing Generative AI for Circuit Board Design
51–60 of 180 posts
Re: Testing Generative AI for Circuit Board Design
#52Earlier quoted context omitted.
I doubt it too, but I notice that I keep underestimating the models. Do you have a challenge task I can try? What's the easiest thing I could get an LLM to do for circuit board design that would surprise you?
Make two separate signals arrive at exactly the same time on two 50 ohm transmission lines that start and end next to each other and go around a right hand bend. At 3.8GHz. Edit: no VSWR constraint. Can add that later :) Edit 2: oh or design a board for a simple 100Mohm input instrumentation amplifier which knows what a guard ring is and how badly the solder mask will screw it up :)
Right - LLMs would be a bit silly for these cases. Both overkill and underkill. Current approach for length matching is throw it off to a domain specific solver. Example test-circuit: https://x.com/DuncanHaldane/status/1803210498009342191
How exact is exactly the same time? Current solver matches to under 10fs, and I think at that level you'd have to fab it to see how close you get with fiber weave skew and all that.
Do you have a test case for a schematic design task?
Re: Testing Generative AI for Circuit Board Design
#53To recycle a rant, there's a whole bunch of hype and investor money riding on a very questionable idea here, namely:
"If we make a really really good specialty text-prediction engine, it could be able to productively mimic an imaginary general AI, and if it can do that then it can productively mimic other specialty AIs, because it's all just intelligence, right?"
Re: Testing Generative AI for Circuit Board Design
#54The conclusions are very optimistic given the results. The LLMs: * Failed to properly understand and respond to the requirements for component selection, which were already pretty generic. * Succeeded in parsing the pinout for an IC but produced an incomplete footprint with incorrect dimensions. * Added extra components to a parsed reference schematic. * Produced very basic errors in a description of filter topologie…
I think the only bit that looked handy in there would be if it could parse PDF datasheets and help you sort them by some hidden parameter. If I give it 100 datasheets for microphones it really should be able to sort them by mechanical height. Maybe I'm too optimistic. The number of times I've had to entirely redo a circuit because of one misplaced connection, yeah, none of those circuits worked for any price before I…
Agree that PDF digesting was the most useful.
I think Gemini could definitely do that microphone study. Good test case! I remember spending 8 hours on DigiKey in the bad old times, looking for an audio jack that was 0.5mm shorter.
Re: Testing Generative AI for Circuit Board Design
#55This feels like an excellent demonstration of the limitation of zero-shot LLMs. It feels like the wrong way to approach this. I'm no expert in the matter, but for "holistic" things (where there are a lot of cross-connections and inter-dependencies) it feels like a diffusion-based generative structure would be better-suited than next-token-prediction. I've felt this way about poetry-generation, and I feel like it migh…
My gut agrees with you that LLMs shouldn't do this well on a specialty domain. But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later. Overall not sure what to expect, but fine tuning experiments would be interesting regardless.
We have 0 y/o/y progress on Advent of Code, for example. Maybe we'll have some progress 6 months from now :) https://www.themotte.org/post/797/chatgpt-vs-advent-of-code
Re: Testing Generative AI for Circuit Board Design
#56I'm terrified that JITX will get into the LLM / Generative AI for boards business. (Don't make me homeless, Duncan!) They are already far ahead of many others with respect to next generation EE CAD. Judicious application of AI would be a big win for them. Edit: adding "TL;DRN'T" to my vocabulary XD
I promise that we want to stay a software company that helps people design things!
Adding Skynetn't to company charter...
Re: Testing Generative AI for Circuit Board Design
#57Earlier quoted context omitted.
I doubt it'd work any better. Most of EE time I have spent is swearing at stuff that looked like it'd work on paper but didn't due to various nuances. I have my own library of nuances but how would you even fine tune anything to understand the black box abstraction of an IC to work out if a nuance applies or not between it and a load or what a transmission line or edge would look like between the IC and the load? Thi…
Really? Most of the time? I find I spend an enormous amount of time on boring stuff like connecting VCC and ground with appropriate decoupling caps, tying output pins from one IC to the input pins on the other, creating library parts from data sheets, etc. There's a handful of interesting problems in any good project where the abstraction breaks down and you have to prove your worth. But a ton of time gets spent on t…
I've found that for speeding up design generation like that, most of the utility comes from the coding approach.
AI can't do it itself (yet), and having it call the higher level functions doesn't save that much time...
Re: Testing Generative AI for Circuit Board Design
#58Earlier quoted context omitted.
My gut agrees with you that LLMs shouldn't do this well on a specialty domain. But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later. Overall not sure what to expect, but fine tuning experiments would be interesting regardless.
> But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later. Heh. This is very true. I think perhaps the thing I'm most amazed by is that simple next-token prediction seems to work unreasonably well for a great many tasks. I just don't know how well that will scale into more complex tasks. With s…
> If you present a simple chess board to an LLM or a complex board to an LLM and ask it to generate the next move, it always responds in the same amount of time.
Is that true, especially if you ask it to think step-by-step?
I would think the model has certain associations for simple/common board states and different ones for complex/uncommon states, and when you ask it to think step-by-step it will explain the associations with a particular state. That "chattiness" may lead it to using more computation for complex boards.
Re: Testing Generative AI for Circuit Board Design
#59Earlier quoted context omitted.
My gut agrees with you that LLMs shouldn't do this well on a specialty domain. But I think there's also the bitter lesson to be learned here: many times people say LLMs won't do well on a task, they are often surprised either immediately or a few months later. Overall not sure what to expect, but fine tuning experiments would be interesting regardless.
We have 0 y/o/y progress on Advent of Code, for example. Maybe we'll have some progress 6 months from now :) https://www.themotte.org/post/797/chatgpt-vs-advent-of-code
Have you tried using more 4000x more samples?
https://redwoodresearch.substack.com/p/getting-50-sota-on-ar...
Re: Testing Generative AI for Circuit Board Design
#60This feels like an excellent demonstration of the limitation of zero-shot LLMs. It feels like the wrong way to approach this. I'm no expert in the matter, but for "holistic" things (where there are a lot of cross-connections and inter-dependencies) it feels like a diffusion-based generative structure would be better-suited than next-token-prediction. I've felt this way about poetry-generation, and I feel like it migh…
> This feels like an excellent demonstration of the limitation of zero-shot LLMs. It feels like the wrong way to approach this. There is one posted on HN every week. How many more do we need to accept the fact this tech is not what it is sold at and we are bored waiting for it get good? I am not say "get better", because it keeps getting better, but somehow doesn't get good.
That’s a perception and the problem isn’t the AI it’s human nature: 1. every time AI is able to do a thing we move the goalposts and say, yeah, but it can’t do that other thing over there; 2. We are impatient, so our ability to get bored tends to outpace the rate of change.