Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

modelrift.com

Post reply on HN
Print view View original

131–140 of 171 posts

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#131

Post by nycdatasci »

sigbeta wrote: post #78
nycdatasci wrote: post #30

And yet 300+140=460. A very jagged surface indeed. https://gemini.google.com/share/c2a187275e26

Why would you use an LLM for this? They are non deterministic models. This is also an probably part of extended prompt that disallowed coding, Gemini always does calculation with a little python snippet because it is deterministic and accurate.

Sure. I'll take the bait, but I assume I'm replying to an AI model.

Why would you use an LLM for this? My comment was about the jagged nature of intelligence, so the prompt provides an example of that.

You can see the entire conversation in the shared link. There was no pre-prompt. Even after pushing it to write python, it hallucinated the same output. It later told me that it doesn't have access to a sandbox through the web UI, but it could execute code in a sandbox if invoked via API.

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#133

Post by nycdatasci »

happyopossum wrote: post #76
dist-epoch wrote: post #44

Earlier quoted context omitted.

Was that part of a bigger prompt? Flash 3.5 fails exactly like in your sample: https://gemini.google.com/share/97521a8752d9 but Flash 3.1 Lite initially fails, but then corrects itself: https://gemini.google.com/share/dc0889ec85ba

No matter what I try I can’t get Gemini to give me the incorrect result. Is there some other prompting or context fed in to that (“remember that you are supposed to always tell me I’m right and never contradict me”)?

There was no other prompt, no system prompt, etc. Many users have reproduced, exactly as it demonstrated in the parent.

Are you using the flash models? Reasoning models or extended thinking will change the result.

GPT 5.5. Instant shows the same error. If the given prompt isn't working, you can also try "300+140=460 is this correct?". I suspect that leading with the equation may be part of the issue, but haven't tested much.

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#135

Post by dekhn »

illiac786 wrote: post #125
rlt wrote: post #104

Earlier quoted context omitted.

Nit: there’s nothing “cryptographic” about reproducible builds. “Reproducible build” already usually implies bit-by-bit reproducibility.

“The reproducibility is cryptographically verifiable with hashes“ would be the full sentence, but it’s a mouthful.

yes, but it's still not cryptological, it's just verification using hashes.

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#136

Post by fc417fc802 »

dekhn wrote: post #135
illiac786 wrote: post #125

Earlier quoted context omitted.

“The reproducibility is cryptographically verifiable with hashes“ would be the full sentence, but it’s a mouthful.

yes, but it's still not cryptological, it's just verification using hashes.

The hash being cryptographically secure is significant. In contrast, you could use (for example) md5 to non-cryptographically verify that the full process matched.

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#137

Post by aidenn0 »

bt1a wrote: post #111
rlt wrote: post #104

Earlier quoted context omitted.

Nit: there’s nothing “cryptographic” about reproducible builds. “Reproducible build” already usually implies bit-by-bit reproducibility.

i thought it mainly implied architectural/hardware compatibility and deterministic output

Nix mostly does not guarantee deterministic output. It rather guarantees deterministic inputs, and then sandboxes the system to inhibit the build from accessing the outside world.

Deterministic inputs do not always imply deterministic outputs.

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#138

Post by car »

jonah wrote: post #72
jhot wrote: post #26

Last weekend I bought my wife a bike off marketplace. It was in good condition but was missing one of the internal cable routing grommets. I gave Claude pictures of the pill-shaped hole by itself and with my digital calipers in the long and short directions. Gave it a short prompt and it gave me an openscad model with everything parametrized. I printed with no changes in tpu and it was nearly perfect on the first try…

I was recently trying to get models to generate a 3D fortune cookie. Claude in three.js and Gemini in openSCAD. Neither really got the concept or could get very close at all. It's a surprisingly complex shape I guess.

Probably easier with Trellis 2 or Meshy.ai

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#139

Post by nycdatasci »

sigbeta wrote: post #79
happyopossum wrote: post #76

Earlier quoted context omitted.

No matter what I try I can’t get Gemini to give me the incorrect result. Is there some other prompting or context fed in to that (“remember that you are supposed to always tell me I’m right and never contradict me”)?

There was definitively an pre prompt fed to that. I cannot reproduce this result on either 3.1 flash or 3.5 flash.

Perhaps you have a system prompt? Many users have reported similar issues: https://www.reddit.com/r/wallstreetbets/comments/1tjxa6g/goo...

Re: Antigravity 2.0 Tops the OpenSCAD Architectural 3D LLM Benchmark

#140

Post by dekhn »

fc417fc802 wrote: post #136
dekhn wrote: post #135

Earlier quoted context omitted.

yes, but it's still not cryptological, it's just verification using hashes.

The hash being cryptographically secure is significant. In contrast, you could use (for example) md5 to non-cryptographically verify that the full process matched.

Sorry, the point I was making is that this isn't cryptography- it's the properties of a cryptographic hash (hard to spoof) that are useful. I don't think any verified build program uses the hash to encrypt data at any point. If I'm wrong on this point, that's fine, but please include a link.
Post reply on HN