Live data from Hacker News

Teaching LLMs how to solid model

willpatrick.xyz

61–70 of 115 posts

Re: Teaching LLMs how to solid model

#61
post #21

The future: "and I want a 3mm hole in one side of the plate. No the other side. No, not like that, at the bottom. Now make it 10mm from the other hole. No the other hole. No, up not sideways. Wait, which way is up? Never mind, I'll do it myself." I'm having trouble understanding why you would want to do this. A good interface between what I want and the model I will make is to draw a picture, not write an essay. This…

If there I in AI then it would tell me where to put the hole and why.

Re: Teaching LLMs how to solid model

#62
post #38

Earlier quoted context omitted.

I have come across a significant number of non engineers wanting to do, what ultimately involves some basic CAD modelling. Some can stall on such tasks for years (home renovation) or just don't do it at all. After some brief research, the main cause is not wanting to sink over 30 hours into learning basics of a cad package of choice. For some reason they imagine it as a daunting, complicated, impenetrable task with m…

>> I have come across a significant number of non engineers wanting to do, what ultimately involves some basic CAD modelling. I very much want Solvespace to be the tool for those people. It's very easy to learn and do the basics. But some of the bugs still need to get fixed (failures tend to be big problems for new users because without experience its hard to explain what's going wrong or a workaround) and we need a…

> I very much want Solvespace to be the tool for those people.

Probably not. "Copyright 2008-2022 SolveSpace contributors. Most recent update June 2 2022."

Re: Teaching LLMs how to solid model

#63
post #38
post #21

The future: "and I want a 3mm hole in one side of the plate. No the other side. No, not like that, at the bottom. Now make it 10mm from the other hole. No the other hole. No, up not sideways. Wait, which way is up? Never mind, I'll do it myself." I'm having trouble understanding why you would want to do this. A good interface between what I want and the model I will make is to draw a picture, not write an essay. This…

I have come across a significant number of non engineers wanting to do, what ultimately involves some basic CAD modelling. Some can stall on such tasks for years (home renovation) or just don't do it at all. After some brief research, the main cause is not wanting to sink over 30 hours into learning basics of a cad package of choice. For some reason they imagine it as a daunting, complicated, impenetrable task with m…

There's a mindset change needed to use a feature tree based constructive solid geometry system. The order in which you do things is implicit in the feature tree. Once you get this, it's not too hard. But figuring out where to start can be tough.

I miss the TechShop days, from when the CEO of Autodesk liked the maker movement and supplied TechShop with full Autodesk Inventor. I learned to use it and liked it. You can still get Fusion 360, but it's not as good.

The problem with free CAD systems is that they suffer from the classic open source disease - a terrible user interface. Often this is patched by making the interface scriptable or programmable or themeable, which doesn't help. 3D UI is really, really hard. You need to be able to do things such as change the viewpoint and zoom without losing the current selection set, using nothing but a mouse.

(Inventor is overkill for most people. You get warnings such as "The two gears do not have a relatively prime number of teeth, which may cause uneven wear.")

Re: Teaching LLMs how to solid model

#64
post #52

> To my surprise, Zoo’s API didn’t perform particularly well in comparison to LLMs generating STLs by creating OpenSCAD This is interesting. As foundational models get better and better, does having proprietary data lose its defensibility more?

Zoo co-founder here. Our product is still pre-v1. But getting to v1 very soon. We actually built a whole new CAD kernel from the ground up. I say this because we can't actually train on models the CAD engine does not yet support. Just 2 weeks ago we shipped csg boolean operations to the CAD engine. This unlocked new data to train our model on that use those operations. So its actually fair to say at the time he used our model we using about 2% of the data we actually have. Once we can use more and more the ability of the model will only get better.

Re: Teaching LLMs how to solid model

#65
post #21

The future: "and I want a 3mm hole in one side of the plate. No the other side. No, not like that, at the bottom. Now make it 10mm from the other hole. No the other hole. No, up not sideways. Wait, which way is up? Never mind, I'll do it myself." I'm having trouble understanding why you would want to do this. A good interface between what I want and the model I will make is to draw a picture, not write an essay. This…

Most likely you won’t be asking for specific things like “3mm hole 3in from the side”, you’ll say things like “Create a plastic enclosure sized to go under a desk, ok add a usb receptacle opening, ok add flanges with standard screw holes” In the text to CAD ecosystem we talk about matching our language/framework to “design intent” a lot. The ideal interface is usually higher level than people expect it to be.

The problem is that this isn't very useful except for the very earliest ideation stages of industrial design, which hardly need CAD anyway.

Most parts need to fit with something else, usually some set of components. Then there are considerations around draft, moldability, size of core pins, sliders, direction of ejection, wall thickness, coring out, radii, ribs for stiffness, tolerances...

LLMs seem far off from being the right answer here. There is, however, lots to make more efficient. Maybe you could tokenize breps in some useful way and see if transformers could become competent speaking in brep tokens? It's hand-wavy but maybe there's something there.

Mechanical engineers do not try to explain models to each other in English. They gather around Solidworks or send pictures to each other. It is incredibly hard to explain a model in English, and I don't see how a traditional LLM would be any better.

Re: Teaching LLMs how to solid model

#66
post #58

I get that CAD interfaces are terrible - but if I imagine the technological utopia of the future - using the english language as the interface sounds terrible no matter how well you do it. Unless you are paraplegic and speaking is your only means of manipulating the world. I much prefer the direction of sculpting with my hands in VR, pulling the dimensions out with a pinch, snapping things parellel with my fine motor…

I’m not sure that CAD interfaces are terrible, it’s just hard work

Re: Teaching LLMs how to solid model

#67
post #21

The future: "and I want a 3mm hole in one side of the plate. No the other side. No, not like that, at the bottom. Now make it 10mm from the other hole. No the other hole. No, up not sideways. Wait, which way is up? Never mind, I'll do it myself." I'm having trouble understanding why you would want to do this. A good interface between what I want and the model I will make is to draw a picture, not write an essay. This…

If you would use LLM-assisted CAD for real industrial design, you would have to end up by specifying exactly where everything has to go and what size it has to be. But if you are doing that then you may as well make an automated program to convert those specific requirements into a 3D model.

Oh wait, that's CAD.

Cynical take aside, I think this could be quite useful for normal people making simple stuff, and could really help consumer 3D printing have a much larger impact.

Re: Teaching LLMs how to solid model

#68
post #21

The future: "and I want a 3mm hole in one side of the plate. No the other side. No, not like that, at the bottom. Now make it 10mm from the other hole. No the other hole. No, up not sideways. Wait, which way is up? Never mind, I'll do it myself." I'm having trouble understanding why you would want to do this. A good interface between what I want and the model I will make is to draw a picture, not write an essay. This…

Here's how it might work, by analogy to the workflow for image generation: "An aerodynamically curved plastic enclosure for a form-over-function guitar amp." Then you get something with the basic shapes and bevels in place, and adjust it in CAD to fit your actual design goals. Then, "Given this shape, make it easy to injection mold." Then it would smooth out some things a little too much, and you'd fix it in CAD. The…

I appreciate that you have given this some thought, but it is clear that you dont have much or any professional experience in 3D modeling or mechanical design.

For the guitar amp, ok. Maybe that prompt will give you a set of surfaces you can scale for the exterior shell of the amp. Because you will need to scale it, or know exactly the dimensions of your speakers, internal chambers, electronics, I/O, baffles, and where those will all ve relative go eachother. Also...Do you need buttons? Jacks/connectors/other I/O? How and where will the connections be routed to other components? Do you need an internal structure with an external aesthetic shell? Or are you going to somehow mold the whole thing in one piece? Where should the part be split? What kind of fasteners will join the parts and where should they be joined? What material is the shell? Can it be thinner to save weight? Or need ribs or thickness for strength? Where does it need to be strong?

These are the issues from 30 seconds of thinking about this. AI (as suggested) could maybe save me from surfacing an exterior cosmetic cover, given presice constraints and dimensions, but at that point, I may as well just do it myself.

If you have a common, easy, already solved an mechanical design problem (hinge e.g.), then you buy an off the shelf component. For everything else, it is bespoke, and every detail matters. Every problem is a "wine glass full to the brim"

Re: Teaching LLMs how to solid model

#69

Earlier quoted context omitted.

Most likely you won’t be asking for specific things like “3mm hole 3in from the side”, you’ll say things like “Create a plastic enclosure sized to go under a desk, ok add a usb receptacle opening, ok add flanges with standard screw holes” In the text to CAD ecosystem we talk about matching our language/framework to “design intent” a lot. The ideal interface is usually higher level than people expect it to be.

The problem is that this isn't very useful except for the very earliest ideation stages of industrial design, which hardly need CAD anyway. Most parts need to fit with something else, usually some set of components. Then there are considerations around draft, moldability, size of core pins, sliders, direction of ejection, wall thickness, coring out, radii, ribs for stiffness, tolerances... LLMs seem far off from bein…

You may or may not be right, but your arguments sound like echos of what software developers were saying four or five years ago. And four or five years ago, they were right.

Don't dismiss an AI tool just because the first iterations aren't useful, it'll be iterated on faster than you can believe possible.

Re: Teaching LLMs how to solid model

#70
post #14
post #9

I tried this a few months back with claude 3.5 writing cadquery code in cline, with render photos for feedback. I got it to model a few simple things like terraforming mars city fairly nicely. However it still involved a fair bit of coaching. I wrote a simple script to automate the process more but it went off the rails too often. I wonder if the models improved image understanding also lead to better spatial underst…

how did you feedback the rendered photos or was it a manual copy-paste step?

Just a python script to render then api call with a prompt to check if the render looks right.
Post reply on HN