Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Working With AI: A concrete example

htmx.org

Post reply on HN
Print view View original

41–50 of 78 posts

Re: Working With AI: A concrete example

#41

Post by appplication »

thunky wrote: post #26
oulipo2 wrote: post #15

Earlier quoted context omitted.

Exactly, LLM is good at "code inpainting" : define clear structures and goals, and it will fill the boilerplate. But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views. But that's integral to the way it's designed and has been trained, to do a kind of "averaging" which limits it's capacity to explore novel designs

> But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views I disagree. Have a conversation with it about your problem and work through design decisions with it. When I do that, I find it gives me a lot of good ideas. Disclaimer: I'm not working on anything groundbreaking (like most people)

I find I don’t necessarily need or want AI to give me ideas, but I would agree having a conversational back and forth generally yields decent results.

I have found being Socratic in my questions, and trying to get the AI to arrive at my intended design via such conversations supplies the right level of context for properly solving the problem. It’s token intensive, without a doubt, but I find the result is the AI tends to be better equipped to handle the many micro decisions that need to be made along the way.

The contrast to this is I give it a detailed prompt where it then asks questions of me, which also generally works but I find the AI tends to not be as well equipped for decisions it needs to make mid implementation.

It’s not perfect, and maybe not even a good fit for some. I also never know what to think when people tell me their idiosyncratic ways of using AI. Ultimately I think the most effective way is whatever lets you translate the vision in your head into the end result.

Re: Working With AI: A concrete example

#42

Post by altmanaltman »

rst wrote: post #21
jdlshore wrote: post #10

Carson’s experience matches mine: AI is good at analysis and boilerplate, but not good at the kind of critical thinking necessary for good designs. If it were human, I would say that it jumps to solutions to quickly, rather than stepping back to consider the big picture and how everything should fit together to make a cohesive whole. It’s not human, of course, and I think this problem actually relates to the fact tha…

One partial mitigation is to ask it to use plan mode -- and then very carefully review the plan before allowing it to execute.

Okay but that means you already know the plan since you are qualified to review it. So why not just tell it the plan yourself (0-shot) vrs having it guess and you review multiple times (n-shot). Wouldn't the former be more effective everytime?

Re: Working With AI: A concrete example

#43

Post by altmanaltman »

thunky wrote: post #26
oulipo2 wrote: post #15

Earlier quoted context omitted.

Exactly, LLM is good at "code inpainting" : define clear structures and goals, and it will fill the boilerplate. But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views. But that's integral to the way it's designed and has been trained, to do a kind of "averaging" which limits it's capacity to explore novel designs

> But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views I disagree. Have a conversation with it about your problem and work through design decisions with it. When I do that, I find it gives me a lot of good ideas. Disclaimer: I'm not working on anything groundbreaking (like most people)

Sure but you can also google your problem and check what is industry standard/what is the correct way to do things (imo in less time than it takes to go through a conversation).

But the problem is that when you ask ai to solve a problem on its own, its default plan can suck. You can mitigate that by research and context but it doesn't mean the initial problem is solved. But even that requires skill and human judgement (both ai conversation research or traditional research) and a lot of people want to skip that entirely.

Re: Working With AI: A concrete example

#45

Post by nok22kon »

piskov wrote: post #28
thorum wrote: post #5

Interesting read! Creating tests is highlighted as something Claude did well, but it strikes me that all the weaker rejected solutions could have been avoided if it were really good at designing intelligent tests for itself. For example, the first solution “was very specific to the reported bug and wouldn’t have fixed the general case” and the third suggestion “prevented the perfectly valid use of as conversion expre…

As humans we have a concept of viscosity. That resistance, like being in quicksand or a swamp, is how you “easily” identify a code smell, something that needs to be refactored, etc. Part of it is human laziness, part of it some concept of elegance, an itch of being not quite tidy as it can be, etc. LLM, being a tiresome little helper, will gladly output hundreds of lines, hacks, and what have you. I don’t think any a…

have you tried asking?

I've used with great success prompts like "when implementing this feature, did you encounter sections of code that were needlessly complex, that were making it hard for you to work? what would you change in the design/architecture to make it leaner?"

Re: Working With AI: A concrete example

#46

Post by stymaar »

jdlshore wrote: post #10

Carson’s experience matches mine: AI is good at analysis and boilerplate, but not good at the kind of critical thinking necessary for good designs. If it were human, I would say that it jumps to solutions to quickly, rather than stepping back to consider the big picture and how everything should fit together to make a cohesive whole. It’s not human, of course, and I think this problem actually relates to the fact tha…

> I suspect that this is a fundamental limitation of LLMs

I suspect there's also a strong sociological bias at play: LLMs are being made by people who are familiar with coding but aren't software engineers. So they design their RL policies around the idea that the LLM must learn how to code, not that they must learn to design a maintenable piece of software.

Re: Working With AI: A concrete example

#48

Post by librasteve »

varun_ch wrote: post #3

maybe slightly unrelated but the new htmx homepage ( https://four.htmx.org/ ) feels a little ironic, seemingly written with tailwindcss and a full JS ecosystem Astro build system. It also has the ‘vibey’ ‘hypey’ landing page design that’s hard to describe but you’ll find on any web framework, rather than dropping you to docs like the old site. Compared to the original simple HTML site it’s really surprising to see fr…

yeuch … should’ve used https://harcstack.org, like the new https://raku.foundation site

Re: Working With AI: A concrete example

#49

Post by rolisz »

giraffe_lady wrote: post #35
wiremine wrote: post #17

It's a good write up, but it's lacking some details, the most important one is: which Claude model was used? The second issue is: what was tooling and the prompt approach? (To be clear, I have no problem with the premise of the write up. But without some details like this, it's sort of like saying "I had a bad board on my deck, and my tape measure wasn't able to help me remove the nails. What a bad tape measure."

It's more like asking what editor and keyboard layout they use. Highly relevant to the user but you should simply assume someone describing work is using a setup for it they find productive. If you decide to dismiss their output it wouldn't be over these details.

Not quite. Model is extremely important in the quality of results. Harness can also influence things.

Re: Working With AI: A concrete example

#50

Post by librasteve »

recursivedoubts wrote: post #4
varun_ch wrote: post #3

maybe slightly unrelated but the new htmx homepage ( https://four.htmx.org/ ) feels a little ironic, seemingly written with tailwindcss and a full JS ecosystem Astro build system. It also has the ‘vibey’ ‘hypey’ landing page design that’s hard to describe but you’ll find on any web framework, rather than dropping you to docs like the old site. Compared to the original simple HTML site it’s really surprising to see fr…

:) i let a younger person on the core team create the new website for something different it is using astro, we are scaling down the use of tailwind (I wanted to give it a try, but didn't really click with it.) I don't mind someone doing something kind of fun with the website and trying something new out, I know some people don't like it but some people do. All good.

i suppose you have to at least try tailwind if you advocate for LOB … in https://harcstack.org, I have started with https://picocss.com which keeps the HTML squeaky clean. it is open to other Themes down the line and I have not rules tailwind out, but I suspect that it will make me feel dirty when I come to it. in general hArc is able to leverage Raku roles for code decomposition and the optimum design is settling on pinning CSS styles to elements (grid, table, form, etc) and encapsulating them so that changes to one thing do not cascade to another
Post reply on HN