About a year ago I had a 2D drawing of a relatively simple, I uploaded it to chatgpt and asked it to model it in cadquery. It required some coaching and manual post processing but it was able to do it. I have since moved to solvespace since even after using cadquery for years I was spending 50% of the time finding some weird structure to continue my drawing from. Solvespace is simply much more productive for me.
Teaching LLMs how to solid model
11–20 of 115 posts
Re: Teaching LLMs how to solid model
#12I've done this, and printed actual models AIs generated. In my experience Grok does the best job with this - it one shots even the more elaborate designs (with thinking). Gemini often screws up, but it sometimes can (get this!) figure things out if you show it what the errors are, as a screenshot. This in particular gives me hope that some kind of RL loop can be built around this. OpenAI models screw up and can't fix…
You've got to be a bit more specific, those words can all refer to many models.
Typically only the most powerful models are worth a try and even then they feel like they aren't capable enough. This is not surprising: to the best of my knowledge none of the current SOTA models was trained to reason about 3D geometry. With Grok there's just one model: Grok3. With OpenAI I used o1 and o3 (after o3 was released). With Google, the visual feedback was with Gemini Pro 2.5. Deepseek also serves only one model. Where there is a toggle (Grok and Deepseek), "thinking" was enabled.
Re: Teaching LLMs how to solid model
#13I did this a few months ago to make a Christmas ornament. There are some rough edges with the process, but for hobby 3D printing, current LLMs with OpenSCAD is a game-changer. I hadn't touched my 3D printer for years until this project.
https://seanmcloughl.in/3d-modeling-with-llms-as-a-cad-luddi...
Re: Teaching LLMs how to solid model
#14I tried this a few months back with claude 3.5 writing cadquery code in cline, with render photos for feedback. I got it to model a few simple things like terraforming mars city fairly nicely. However it still involved a fair bit of coaching. I wrote a simple script to automate the process more but it went off the rails too often. I wonder if the models improved image understanding also lead to better spatial underst…
how did you feedback the rendered photos or was it a manual copy-paste step?
Re: Teaching LLMs how to solid model
#15Your prompts are very long for how simple the models are, using a CAD package would be far more productive.
I can see AI being used to generate geometry, but not a text based one, it would have to be able to reason with 3d forms and do differential geometry.
You might be able to get somewhere by training an LLM to make models with a DSL for Open Cascade, or any other sufficiently powerful modelling kernel. Then you could train the AI to make query based commands, such as:
// places a threaded hole at every corner of the top surface (maybe this is an enclosure)
CUT hole(10mm,m3,threaded) LOCATIONS surfaces().parallel(Z).first().inset(10).outside_corners()
Re: Teaching LLMs how to solid model
#16Oh, poor itty-bitty Skynet won't be able to create Terminators without mastering CAD modelling, so let us totally teach it
Re: Teaching LLMs how to solid model
#17As a huge OpenSCAD fan and everyday Cursor user, it seems obvious to me that there's a huge opportunity _if_ we can improve the baseline OpenSCAD code quality.
If the model could plan ahead well, set up good functions, pull from standard libraries, etc., it would be instantly better than most humans.
If it had a sense of real-world applications, physics, etc., well, it would be superhuman.
Is anyone working on this right now? If so I'd love to contribute.
Re: Teaching LLMs how to solid model
#18Your prompts are very long for how simple the models are, using a CAD package would be far more productive. I can see AI being used to generate geometry, but not a text based one, it would have to be able to reason with 3d forms and do differential geometry. You might be able to get somewhere by training an LLM to make models with a DSL for Open Cascade, or any other sufficiently powerful modelling kernel. Then you c…
I definitely agree with your point about the long prompts.
The long prompts are primarily an artifact of trying to make an eval where there is a "correct" STL.
I think your broader point, text input is bad for CAD, is also correct. Some combo of voice/text input + using a cursor to click on geometry makes sense. For example, clicking on the surface in question and then asking for "m6 threaded holes at the corners". I think a drawing input also make sense as its quite quick to do.
Re: Teaching LLMs how to solid model
#19Your prompts are very long for how simple the models are, using a CAD package would be far more productive. I can see AI being used to generate geometry, but not a text based one, it would have to be able to reason with 3d forms and do differential geometry. You might be able to get somewhere by training an LLM to make models with a DSL for Open Cascade, or any other sufficiently powerful modelling kernel. Then you c…
There are diffusion models for 3D generation. They make pretty good decorative or ornamental models, like figurines. They are less good for CAD.
Re: Teaching LLMs how to solid model
#20I 3D printed a replacement screw cap for something that GPT-4o designed for me with OpenSCAD a few months ago. It worked very well and the resulting code was easy to tweak.
Good to hear that newer models are getting better at this. With evals and RL feedback loops, I suspect it's the kind of thing that LLMs will get very good at.
Vision language models can also improve their 3D model generation if you give them renders of the output: "Generating CAD Code with Vision-Language Models for 3D Designs" https://arxiv.org/html/2410.05340v2
OpenSCAD is primitive. There are many libraries that may give LLMs a boost. https://openscad.org/libraries.html