Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teaching LLMs how to solid model

willpatrick.xyz

Post reply on HN
Print view View original

81–90 of 115 posts

Re: Teaching LLMs how to solid model

#81

Post by throwup238 »

zonkerdonker wrote: post #68
whatshisface wrote: post #48

Earlier quoted context omitted.

Here's how it might work, by analogy to the workflow for image generation: "An aerodynamically curved plastic enclosure for a form-over-function guitar amp." Then you get something with the basic shapes and bevels in place, and adjust it in CAD to fit your actual design goals. Then, "Given this shape, make it easy to injection mold." Then it would smooth out some things a little too much, and you'd fix it in CAD. The…

I appreciate that you have given this some thought, but it is clear that you dont have much or any professional experience in 3D modeling or mechanical design. For the guitar amp, ok. Maybe that prompt will give you a set of surfaces you can scale for the exterior shell of the amp. Because you will need to scale it, or know exactly the dimensions of your speakers, internal chambers, electronics, I/O, baffles, and whe…

I think you’re jumping too fast to the “vibe CADing” extreme. It’s been a while since I’ve used Solidworks in anger so I’d rather use ECAD as an example: I’d kill for the ability to give Altium a PDF datasheet and have it generate footprints or schematic components tailored to my specific pinout for a microcontroller. Or give it a pdf of routing guidelines and have it convert those to design rules tied just to those nets. Those are the details that take up most of the time (although I’d still spend quite a lot of tine verifying all the output).

In MCAD it’s less of a problem because all the big vendors like Misumi, McMaster, et al have extensions or downloadable models but anything custom could probably benefit from LLMs (I say this as someone who is generally skeptical of their vision capabilities). I don’t think vibe CADing will work because most parts are too parametrized but giving an AI a bunch of PDFs and a size + thickness is probably going to be really productive.

Re: Teaching LLMs how to solid model

#82

Post by acyou »

I think if you could directly tokenize 3D geometry and train an LLM on 3D models directly, you might get somewhere. In order to prompt it, you would need to feed it a 3D model(s), and prompts and it could give you back a different 3D model. This has been done to some extent with generative modeling pre-LLM, but I don't know of any work that takes LLM techniques applied to language and applies them to "tokenizing" 3D geometry. I suspect NVIDIA is probably working very hard on this exact problem for graphics applications.

For mechanical design, 3D modeling is highly integrative, inputs are from a vast array of poorly specified inputs with a high amount of unspecified and fluid contextual knowledge, and outputs are not well defined either. I'm not convinced that mechanical design is particularly well suited to pairing with LLM workflow. Certain aspects, sure. But 3D models and drawings that we consider "well-defined" are still usually quite poorly defined, and from necessity rely heavily on implicit assumptions.

The geometry of machine threads, for example. Are you going to have a big computer specify the position of each of the atoms in the machine thread? Even the most detailed CAD/CAM packages have thread geometry extremely loosely defined, to the point of listing the callout, and not modeling any geometry at all in many cases.

It would just be very difficult to feed enough contextual knowledge into an LLM to have the knowledge it needs to do mechanical design. Therein lies the main problem. And I will stress that it's not a training problem, it's a prompt problem, if that makes sense.

Re: Teaching LLMs how to solid model

#83

Post by flimflamm »

It would be interesting to see how far one could get with fine tuning and RL. One could for example take free scad models, give 2D pictures from different angles and ask LLM to recreate the 3D design in scad language. Then compare volumetrically how close they are and provide feedback.

Once that is done then ask LLM to create a prompt and compare outputs etc..

Re: Teaching LLMs how to solid model

#84

Post by wavefrontbakc »

jillesvangurp wrote: post #75
abe_m wrote: post #31

Earlier quoted context omitted.

I think this is along the lines of the AI horseless carriage[1] topic that is also on the front page right now. You seem to be describing the current method as operated through an AI intermediary. I think the power in AI for CAD will be at a higher level than lines, faces and holes. It will be more along the lines of "make a bracket between these two parts". "Make this part bolt to that other part". "Attach this pump…

It's also going to be about diagnosing issues. "This part broke right here, explain why and come up with a solution", "Evaluate the robustness of this solution", "Can I save some material and reduce the weight", etc. Those are the kind of high level questions that an LLM with a decent understanding of CAD and design might be able to deal with soon and it will help speed up expensive design iterations. A neat trick wi…

I think the cost of mistakes is the major driving force behind where you can adopt tools like these. Generating a picture of a chair with five legs? No big deal. Generating supports for a bridge that'll collapse next week? Big problem.

> It will point out things that are unclear, etc. You can go far beyond just micro managing incremental edits to some thing.

When prompted an LLM will also point it out when it's perfectly clear. LLM is just text prediction, not magic

Re: Teaching LLMs how to solid model

#85

Post by wavefrontbakc »

Nevermark wrote: post #79
jillesvangurp wrote: post #75

Earlier quoted context omitted.

It's also going to be about diagnosing issues. "This part broke right here, explain why and come up with a solution", "Evaluate the robustness of this solution", "Can I save some material and reduce the weight", etc. Those are the kind of high level questions that an LLM with a decent understanding of CAD and design might be able to deal with soon and it will help speed up expensive design iterations. A neat trick wi…

> Mostly the main limitation with LLMs is the imagination of the person using it. Ask the right questions and they get a lot more useful. Completely agree. We get waves of comments on HN downplaying model abilities or their value. Many people don’t seem to explore and experiment with them enough. I have 3 screens. The left one has two models on it. The right one has a model & a web browser for quick searches. I work…

I've only found useful answers for when I've begun something from completely zero. Things I've got even a reasonable grasp on I start to spot large dangerous holes a model very confidently claims is the right solution. Stuff like bringing camping stove inside a tent to avoid rainy weather, nevermind it being suicide at the same time.

>I web search maybe 10% of what I did six months ago.

Me too, though this is more driven by the total cliff-fall of web search result quality

Re: Teaching LLMs how to solid model

#86

Post by krige »

jillesvangurp wrote: post #75
abe_m wrote: post #31

Earlier quoted context omitted.

I think this is along the lines of the AI horseless carriage[1] topic that is also on the front page right now. You seem to be describing the current method as operated through an AI intermediary. I think the power in AI for CAD will be at a higher level than lines, faces and holes. It will be more along the lines of "make a bracket between these two parts". "Make this part bolt to that other part". "Attach this pump…

It's also going to be about diagnosing issues. "This part broke right here, explain why and come up with a solution", "Evaluate the robustness of this solution", "Can I save some material and reduce the weight", etc. Those are the kind of high level questions that an LLM with a decent understanding of CAD and design might be able to deal with soon and it will help speed up expensive design iterations. A neat trick wi…

> Not bad. I'm sorry but it's pretty bad. Hierarchy complaint is bogus, so's navigation overload, it hallucinates BG as white, and the rest is very generic.

Re: Teaching LLMs how to solid model

#87

Post by baq »

wavefrontbakc wrote: post #84
jillesvangurp wrote: post #75

Earlier quoted context omitted.

It's also going to be about diagnosing issues. "This part broke right here, explain why and come up with a solution", "Evaluate the robustness of this solution", "Can I save some material and reduce the weight", etc. Those are the kind of high level questions that an LLM with a decent understanding of CAD and design might be able to deal with soon and it will help speed up expensive design iterations. A neat trick wi…

I think the cost of mistakes is the major driving force behind where you can adopt tools like these. Generating a picture of a chair with five legs? No big deal. Generating supports for a bridge that'll collapse next week? Big problem. > It will point out things that are unclear, etc. You can go far beyond just micro managing incremental edits to some thing. When prompted an LLM will also point it out when it's perfe…

The text of every Nobel winning physics theory was predicted in someone’s head, too

Re: Teaching LLMs how to solid model

#88

Post by ben_w »

wavefrontbakc wrote: post #84
jillesvangurp wrote: post #75

Earlier quoted context omitted.

It's also going to be about diagnosing issues. "This part broke right here, explain why and come up with a solution", "Evaluate the robustness of this solution", "Can I save some material and reduce the weight", etc. Those are the kind of high level questions that an LLM with a decent understanding of CAD and design might be able to deal with soon and it will help speed up expensive design iterations. A neat trick wi…

I think the cost of mistakes is the major driving force behind where you can adopt tools like these. Generating a picture of a chair with five legs? No big deal. Generating supports for a bridge that'll collapse next week? Big problem. > It will point out things that are unclear, etc. You can go far beyond just micro managing incremental edits to some thing. When prompted an LLM will also point it out when it's perfe…

> I think the cost of mistakes is the major driving force behind where you can adopt tools like these. Generating a picture of a chair with five legs? No big deal. Generating supports for a bridge that'll collapse next week? Big problem

Yes, indeed.

But:

Why can LLMs generally write code that even compiles?

While I wouldn't trust current setups, there's no obvious reason why even a mere LLM cannot be used to explore the design space when the output can be simulated to test its suitability as a solution — even in physical systems, this is already done with non-verbal genetic algorithms.

> LLM is just text prediction, not magic

"Sufficiently advanced technology is indistinguishable from magic".

Saying "just text prediction" understates how big a deal that is.

Re: Teaching LLMs how to solid model

#90

Post by littlestymaar »

esperent wrote: post #80
littlestymaar wrote: post #76

Earlier quoted context omitted.

While LLMs are a useful tool for software development, if you try asking them the software equivalent of “Create a plastic enclosure sized to go under a desk, ok add a usb receptacle opening, ok add flanges with standard screw holes” you'll end up with the equivalent of “No the other side. No, not like that, at the bottom. Now make it 10mm from the other hole. No the other hole. No, up not sideways. Wait, which way i…

> Create a plastic enclosure sized to go under a desk, ok add a usb receptacle opening, ok add flanges with standard screw holes With vague specifications like these, you'd get garbage from a human too. What works for software, and I suspect for other technical fields like CAD too, is to treat it like a junior developer who has an extreme breadth of knowledge but not much depth. You will need to take care to clearly…

> With vague specifications like these, you'd get garbage from a human too.

You'll never have better input than this at the beginning of any project from the person that brings the use-case. That's a full job to help them define the needs more accurately. And if you always work with clear specifications it's just because there's someone in front of you that has helped write the spec starting from the loose business requirement.

> You will need to take care to clearly specify your requirements

Yes, but as I discussed above, for such tasks it's going to be very frustrating and less efficient than doing things by yourself. The only reason why you'd accept to go through this kind of effort for an intern is that because you expect him to learn and become autonomous at some point. With current tech, an LLM will forever remain as clueless as it started.

Post reply on HN