Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Jacquard, a programming language for AI-written, human-reviewed code

github.com

Post reply on HN
Print view View original

61–67 of 67 posts

Re: Show HN: Jacquard, a programming language for AI-written, human-reviewed code

#61

Post by orbital-decay »

jondwillis wrote: post #60
orbital-decay wrote: post #18

Earlier quoted context omitted.

The distribution is actually pretty diverse, but this diversity is constrained by mode collapse.

Thanks for the clarification. “Mode collapse” explains so much about how I have been experiencing output convergence and repetition for a while now.

The name is a bit of a misnomer, though. It describes the statistical mode and implies the token selection, but what actually collapses is internal semantic mapping: if you give the model the same input concept, it will nearly always respond with the same or a very similar output concept, even if it has a wide range of valid options to select from. Sometimes it's not even 1:1, but many-to-one, which is what you see in case of the Jacquard collision. Even if you ban/zero some logits, it will try responding with the same idea in synonyms, and be really stubborn in it. The fact that the instruction-tuned model can and will vaguely predict the response before writing it makes longer thinking and longer replies introduce much less entropy than you'd think. Which means even less variance and more baked-in stereotypes.

Re: Show HN: Jacquard, a programming language for AI-written, human-reviewed code

#62

Post by derdi »

zparky wrote: post #57
derdi wrote: post #56

Earlier quoted context omitted.

Just in case the above is not considered constructive enough, how about this: In my experience, with some prompting and iterations, AIs do reasonably well at writing solid, "boring" code. They do really badly at writing useful documentation, and this is a major pain point when reviewing their output. A system that claims to be oriented towards human review must address this somehow. It must talk about how it addresse…

I always feel like I'm losing my mind when reading LLM comments or prose for code. It's so esoteric and 'i-am-very-smart' and makes me feel like I'm not supposed to read the comments in the first place

Once, during an especially tedious review session, I told my agent something like "comments are not meant to demonstrate how well you understand the system, they are meant to help others understand the system". It felt like that helped.

Re: Show HN: Jacquard, a programming language for AI-written, human-reviewed code

#64

Post by wren6991 »

imtringued wrote: post #55
wren6991 wrote: post #4

Given how poorly LLMs do with writing prompts for LLMs, I'm not sure I'd trust their judgement in designing a language for LLMs. > and the runtime requires explicit permission to touch the filesystem, network, etc This feels like more of an OS problem (or library problem) than a language problem. > Run one program against many worlds. The same code can run against the real network, a scripted fake, a recording of las…

>This feels like more of an OS problem (or library problem) than a language problem. And from my perspective it's the exact opposite. You can apply a sandbox on the OS level and the library you call will crash at runtime. In your model you now get to harass the library author "hey you didn't tell me you do I/O" after it is already too late.

Crashing instead of raising an exception or bubbling a return code sounds like a legitimate bug ticket IMO.

One reason I'm in favour of handling capabilities in the OS is so we can stop having trivial symlink traversal and /.. traversal bugs in path filters (or indeed more complex bugs in the face of FS-specific linking primitives).

Re: Show HN: Jacquard, a programming language for AI-written, human-reviewed code

#67

Post by zparky »

derdi wrote: post #62
zparky wrote: post #57

Earlier quoted context omitted.

I always feel like I'm losing my mind when reading LLM comments or prose for code. It's so esoteric and 'i-am-very-smart' and makes me feel like I'm not supposed to read the comments in the first place

Once, during an especially tedious review session, I told my agent something like "comments are not meant to demonstrate how well you understand the system, they are meant to help others understand the system". It felt like that helped.

that sounds like a good idea to try, thanks!
Post reply on HN