Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Working With AI: A concrete example

htmx.org

Post reply on HN
Print view View original

51–60 of 78 posts

Re: Working With AI: A concrete example

#51

Post by jacobedawson »

jdlshore wrote: post #10

Carson’s experience matches mine: AI is good at analysis and boilerplate, but not good at the kind of critical thinking necessary for good designs. If it were human, I would say that it jumps to solutions to quickly, rather than stepping back to consider the big picture and how everything should fit together to make a cohesive whole. It’s not human, of course, and I think this problem actually relates to the fact tha…

I feel as though that world model strongly correlates with memory - the experience of having jumped to a conclusion early and full-steaming ahead, only to be bitten by constraints and problems later down the track.

Part of that is critical thinking and projecting forward / simulating potential issues, and part of that is that memory which in humans we probably would see as "wisdom".

I don't know if that's a fundamental limitation of LLMs, or, rather, that this can be solved moving forward with better memory systems, harnesses, and context windows.

Re: Working With AI: A concrete example

#52

Post by mjfisher »

jdlshore wrote: post #34
rst wrote: post #21

Earlier quoted context omitted.

One partial mitigation is to ask it to use plan mode -- and then very carefully review the plan before allowing it to execute.

My experience with AI plans is that they’re a wall of text that’s very hard to extract meaning from. Combined with it not doing a good job to begin with, I don’t think plan+revise is a great use of time.

That's interesting and actually the opposite of mine. I wonder if it's stack or methodology dependant? For reference I'm usually using cursor and opus4.6 and for a bigger piece of work:

- Start in ask mode - "I'm planning on doing X to achieve Y; are there any alternative approaches? What problems might I run into?"

- Chat for a bit and get the high level approach, switch to plan mode and ask for a nicely formatted plan

- What's kicked out is already in the rough shape of the discussion so far, so it's a case of following a nicely formatted doc through and highlighting sections of text and asking for clarification or changes

- Hitting "build" and then reviewing what's been done

For a new service I might spend an hour in ask/plan mode - but then it gets 95% of the build itself right first time.

Do you do the same with different results, or is there a different stack/methodology you go through?

Re: Working With AI: A concrete example

#53

Post by fugaziboutit »

weakfish wrote: post #37
jdlshore wrote: post #34

Earlier quoted context omitted.

My experience with AI plans is that they’re a wall of text that’s very hard to extract meaning from. Combined with it not doing a good job to begin with, I don’t think plan+revise is a great use of time.

I feel the same way. Maybe it’s the ADHD, maybe I’m just dumb, but I cannot parse well the giant walls they tend to produce.

An LLM conversation is like handling clay. When I don't grok an answer I mold the LLM's approach to fit my level of mastery of the subject. It's one of the few interactions you can have in life where you can tell someone how to talk to you without considering how they feel about being ordered around.

Re: Working With AI: A concrete example

#54

Post by epolanski »

jdlshore wrote: post #10

Carson’s experience matches mine: AI is good at analysis and boilerplate, but not good at the kind of critical thinking necessary for good designs. If it were human, I would say that it jumps to solutions to quickly, rather than stepping back to consider the big picture and how everything should fit together to make a cohesive whole. It’s not human, of course, and I think this problem actually relates to the fact tha…

In my experience harness can do wonders to improve this.

Instead of asking it to generically to analyze and do X, you can use brainstorming skills like those from superpowers [1].

This makes it approach the problem better and keeps you in the loop.

Another step is then to have it review its plans by another LLM acting doing adversarial review. I have a claude skill [2] that calls codex to do it, and they chat among each other.

It's a tremendous boost in design quality.

[1] https://github.com/obra/Superpowers

[2] https://gist.github.com/enricopolanski/6c5038a8e20cc4098cd99...

Re: Working With AI: A concrete example

#55

Post by mewpmewp2 »

jacobedawson wrote: post #51
jdlshore wrote: post #10

Carson’s experience matches mine: AI is good at analysis and boilerplate, but not good at the kind of critical thinking necessary for good designs. If it were human, I would say that it jumps to solutions to quickly, rather than stepping back to consider the big picture and how everything should fit together to make a cohesive whole. It’s not human, of course, and I think this problem actually relates to the fact tha…

I feel as though that world model strongly correlates with memory - the experience of having jumped to a conclusion early and full-steaming ahead, only to be bitten by constraints and problems later down the track. Part of that is critical thinking and projecting forward / simulating potential issues, and part of that is that memory which in humans we probably would see as "wisdom". I don't know if that's a fundament…

Yeah, I think it's more so learning from experiences that didn't scale. E.g. I started out with Notepad, and wrote everything for the website I wanted to build in a single large massive php file. I of course don't do that anymore, but it was a step by step iterative progress to move to where I am now. Although I still miss how easy it was to see changes locally, deploy to prod quickly, and make hotfixes in prod. I sometimes think maybe I should go back to php.

Re: Working With AI: A concrete example

#56

Post by internet_points »

librasteve wrote: post #48
varun_ch wrote: post #3

maybe slightly unrelated but the new htmx homepage ( https://four.htmx.org/ ) feels a little ironic, seemingly written with tailwindcss and a full JS ecosystem Astro build system. It also has the ‘vibey’ ‘hypey’ landing page design that’s hard to describe but you’ll find on any web framework, rather than dropping you to docs like the old site. Compared to the original simple HTML site it’s really surprising to see fr…

yeuch … should’ve used https://harcstack.org , like the new https://raku.foundation site

nah, the thlh stack (tailwind+htmx+lucid+haskell) is much more unpronouncable https://github.com/monadicsystems/haskell-htmx-examples

Re: Working With AI: A concrete example

#57

Post by piskov »

nok22kon wrote: post #45
piskov wrote: post #28

Earlier quoted context omitted.

As humans we have a concept of viscosity. That resistance, like being in quicksand or a swamp, is how you “easily” identify a code smell, something that needs to be refactored, etc. Part of it is human laziness, part of it some concept of elegance, an itch of being not quite tidy as it can be, etc. LLM, being a tiresome little helper, will gladly output hundreds of lines, hacks, and what have you. I don’t think any a…

have you tried asking? I've used with great success prompts like "when implementing this feature, did you encounter sections of code that were needlessly complex, that were making it hard for you to work? what would you change in the design/architecture to make it leaner?"

Everything is just one more prompt away, I swear — literally like a gambling addict with a slot machine.

You forgot the premise of the article and why the proposed solutions were not good. It was not the complexity of a solution: they were simple fast fixes like a tape on a leak, but the hacky tape they were.

(of course I tried, the code after “refactor” is still shit unless you start going very explicit about it at a point of being better and faster of doing it yourself)

Re: Working With AI: A concrete example

#59

Post by Michelangelo11 »

> Technical debt, I assert without evidence1, grows exponentially, and therefpre it is very important to minimize it in your projects.

This actually seems like a really important idea absolutely deserving of its own blog post.

I'd have to think about the exact argument for why this feels so right, but the kernel would go something like this: whatever you build on those parts of the codebase where you have technical debt incurs new technical debt, because you're building on top of abstractions you'll remove later. The reason you have to remove the new abstractions, too, is that abstractions are like puzzle pieces: their structure determines which other abstractions they can connect with. So, as a rule (there are some exceptions), you can't take out one bad part, replace it with another, and leave everything around it untouched.

And, of course, it's easier to build on top of something creaky but currently serviceable than it would be to first rip that out and replace it, so that's what you do in most cases ... and the whole codebase gets more creaky and less serviceable; you increase the amount of abstractions you'd have to rip out and replace before building something new. The problem does, indeed, grow exponentially.

The argument is free to a good home -- I don't have the time for a full, meticulous elaboration, but I'd love to read one if someone is interested in making it.

Re: Working With AI: A concrete example

#60

Post by nok22kon »

piskov wrote: post #57
nok22kon wrote: post #45

Earlier quoted context omitted.

have you tried asking? I've used with great success prompts like "when implementing this feature, did you encounter sections of code that were needlessly complex, that were making it hard for you to work? what would you change in the design/architecture to make it leaner?"

Everything is just one more prompt away, I swear — literally like a gambling addict with a slot machine. You forgot the premise of the article and why the proposed solutions were not good. It was not the complexity of a solution: they were simple fast fixes like a tape on a leak, but the hacky tape they were. (of course I tried, the code after “refactor” is still shit unless you start going very explicit about it at…

yes, LLMs are not perfect

this is what separates real engineers which solve hard problems, adapt and overcome, versus the ones which complain that whatever they have access to is not perfect and so they will give up on it because "its unusable"

Post reply on HN