Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why agents do not write most of our code – A reality check

octomind.dev

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Why agents do not write most of our code – A reality check

#11

Post by pietz »

reaslonik wrote: post #6

One thing I find that constantly makes pain for users is assuming that any of these models are thinking, when in reality they're completing a sentence. This might seem like a nitpick at first, but it's a huge deal in reality: if you ask a language model to evaluate whether a solution is right, it's not evaluating the solution, it's giving you a statistically likely next sentence where yes and no are fairly common. If…

Can you go into a bit more detail why the two approaches are so different in your opinion?

I don't think I agree and I want to understand this argument better.

Re: Why agents do not write most of our code – A reality check

#13

Post by ismailmaj »

pietz wrote: post #11
reaslonik wrote: post #6

One thing I find that constantly makes pain for users is assuming that any of these models are thinking, when in reality they're completing a sentence. This might seem like a nitpick at first, but it's a huge deal in reality: if you ask a language model to evaluate whether a solution is right, it's not evaluating the solution, it's giving you a statistically likely next sentence where yes and no are fairly common. If…

Can you go into a bit more detail why the two approaches are so different in your opinion? I don't think I agree and I want to understand this argument better.

I’m guessing the argument is that LLMs get worse for problems they haven’t seen before, so you may assume they think for problems that are commonly discussed in the internet or seen on github, but once you step out of that zone, you get plausible but logically false results.

That or a reductive fallacy, in either case I’m not convinced, IMO they are just not smart enough (either due to lack of complexity in the architecture or bad training that didn’t help it generalize reasoning patterns).

Re: Why agents do not write most of our code – A reality check

#14

Post by graphememes »

Every time I read a post about this, none of the prompts are shared, and when I review the actual commands and how the AI is working it makes me realize that the person who is driving is not experienced in doing so. AI's will do a best attempt, you can see this by looking at the reasoning / thinking output, additionally, the temperature, is usually pretty moderate (4.5-8) and so you'll have heavy "creative liberties" taken. So you need to account for that, you have to show it the right and wrong way to do things. I don't usually use agents or AI for things that are one-offs but not copy & paste, or for deep thinking / critical tasks that require human thought where AI wouldn't be able to do it.

For all the other trivial things, I can delegate those out to it, and expect junior results when I give it sub-optimal guidance, however through nominal and or extreme guidance I can get adequate / near-perfect results.

Another dimension that really matters here is the actual model used, not every model is the same.

Also, if the AI does something wrong, have it assess why things went wrong, revert back to the previous checkpoint and integrate that into the plan.

You're driving, you are ultimately, in control, learn to drive. It's a tool, it can be adjusted, you can modify the output, you can revert, you can also just not use it. But, if you do actually learn how to use it you'll find it can speed up your process. It is not a cure-all though, it's good in certain situations, just like a hammer.

Re: Why agents do not write most of our code – A reality check

#15

Post by ben_w »

giuscri wrote: post #9
reaslonik wrote: post #6

One thing I find that constantly makes pain for users is assuming that any of these models are thinking, when in reality they're completing a sentence. This might seem like a nitpick at first, but it's a huge deal in reality: if you ask a language model to evaluate whether a solution is right, it's not evaluating the solution, it's giving you a statistically likely next sentence where yes and no are fairly common. If…

but it’s also true that the next sentence is generated by evaluating the whole conversation including the proposed solution. my mental model is that the llm learned to predict what another person would say just by looking at that solution. so it’s really telling whether the solution is likely (likely!) to be right or wrong

Slight quibble, but the reinforcement learning from human feedback means they're trained (somewhat) on what the specific human asking the question is likely to consider right or wrong.

This is both why they're sycophantic, and also why they're better than just median internet comments.

But this is only a slight quibble, because what you say is also somewhat true, and why they have such a hard time saying "I don't know".

Re: Why agents do not write most of our code – A reality check

#16

Post by raflueder »

I had a similar experience a couple of months ago where I decided to give it a go and "vibe code" a small TUI to get a feel for the workflow.

I used Claude Code and while the end result works (kinda) I noticed I was less satisfied with the process and, more importantly, I now had to review "someone else's" code instead of writing it myself, I had no idea of the internal workings of the application and it felt like starting at day one on a new codebase. It shifted my way of working from thinking/writing into reviewing/giving feedback which for me personally is way less mentally stimulating and rewarding.

There were def. some "a-ha" moments where CC came up with certain suggestions I wouldn't have thought of myself but those were only a small fraction of the total output and there's def. a dopamine hit from seeing all that code being spit out so fast.

Used as a prototyping tool to quickly test an idea seems to be a good use case but there should be better tooling around taking that prototype, splitting it into manageable parts, sharing the reasoning behind it so I can then rework it so I have the necessary understanding to move it forward.

For now I've decided to stick to code completion, writing of unit tests, commit messages, refactoring short snippets, CHANGELOG updates, it does fairly well on all of those small very focused tasks and the saved time on those end up being net positive.

Re: Why agents do not write most of our code – A reality check

#17

Post by giuscri »

ben_w wrote: post #15
giuscri wrote: post #9

Earlier quoted context omitted.

but it’s also true that the next sentence is generated by evaluating the whole conversation including the proposed solution. my mental model is that the llm learned to predict what another person would say just by looking at that solution. so it’s really telling whether the solution is likely (likely!) to be right or wrong

Slight quibble, but the reinforcement learning from human feedback means they're trained (somewhat) on what the specific human asking the question is likely to consider right or wrong. This is both why they're sycophantic, and also why they're better than just median internet comments. But this is only a slight quibble, because what you say is also somewhat true, and why they have such a hard time saying "I don't kno…

idk… maybe we’ll found out the reason is that on the internet no one ends a conversation saying “i don’t know” :D

Re: Why agents do not write most of our code – A reality check

#18

Post by mnky9800n »

raflueder wrote: post #16

I had a similar experience a couple of months ago where I decided to give it a go and "vibe code" a small TUI to get a feel for the workflow. I used Claude Code and while the end result works (kinda) I noticed I was less satisfied with the process and, more importantly, I now had to review "someone else's" code instead of writing it myself, I had no idea of the internal workings of the application and it felt like st…

> Used as a prototyping tool to quickly test an idea seems to be a good use case but there should be better tooling around taking that prototype, splitting it into manageable parts, sharing the reasoning behind it so I can then rework it so I have the necessary understanding to move it forward.

This would be amazing. I think claude code is a great prototyping tool, but I agree, you don't really learn your code base. But I think, that is okay for a prototype if you just want to see if the idea works at all. Then you can restart as you say with some scaffolding to implement it better.

Re: Why agents do not write most of our code – A reality check

#19

Post by nijave »

reaslonik wrote: post #6

One thing I find that constantly makes pain for users is assuming that any of these models are thinking, when in reality they're completing a sentence. This might seem like a nitpick at first, but it's a huge deal in reality: if you ask a language model to evaluate whether a solution is right, it's not evaluating the solution, it's giving you a statistically likely next sentence where yes and no are fairly common. If…

>The only way to use a tool like this is to give a problem that fits context

Or give context to the model which fits the problem. That's more of an art than a science at this point it seems

I think people with better success are those better at generating prompts but that's non trivial

Re: Why agents do not write most of our code – A reality check

#20

Post by nijave »

pietz wrote: post #11
reaslonik wrote: post #6

One thing I find that constantly makes pain for users is assuming that any of these models are thinking, when in reality they're completing a sentence. This might seem like a nitpick at first, but it's a huge deal in reality: if you ask a language model to evaluate whether a solution is right, it's not evaluating the solution, it's giving you a statistically likely next sentence where yes and no are fairly common. If…

Can you go into a bit more detail why the two approaches are so different in your opinion? I don't think I agree and I want to understand this argument better.

They regurgitate what they're trained on so they're largely consensus based. However, the consensus can be frequently wrong--especially when the information is outdated

Someone with the ability to "think" should be able to separate oft repeated fiction from fact

Post reply on HN