Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vibe coding and agentic engineering are getting closer than I'd like

simonwillison.net

Post reply on HN
Print view View original

351–360 of 958 posts

Re: Vibe coding and agentic engineering are getting closer than I'd like

#351

Post by user34283 »

mkozlows wrote: post #91
gabriela_c wrote: post #13

Claude often does things in more detail, and even better, than I would, in the first pass. But I don't understand how anybody stands comments generated by an LLM? It's seriously the thing that worries (and bothers) me the most. I almost never let unedited LLM comments pass. At a minimum. Most of the time, I use my own vibe-coded tool to run multiple GitHub-PR-review-style reviews, and send them off to the agent to ma…

The comments aren't an LLM thing, they're a Claude thing. Codex doesn't write those gross hyper-verbose comments.

In my experience Codex barely writes any comments, despite my attempts to encourage it in the AGENTS.md.

Re: Vibe coding and agentic engineering are getting closer than I'd like

#352

Post by chromacity »

hintymad wrote: post #302
zarzavat wrote: post #24

Perhaps I've missed a few weeks worth of progress, but I don't think that AIs have become more trustworthy, the errors are just more subtle. If the code doesn't compile, that's easy to spot. If the code compiles but doesn't work, that's still somewhat easy to spot. If the code compiles and works, but it does the wrong thing in some edge case, or has a security vulnerability, or introduces tech debt or dubious archite…

> I don't think that AIs have become more trustworthy, the errors are just more subtle. Honest question: what about the counter-argument that humans make subtle mistakes all the time, so why do we treat AI any differently? A difference to me is that when we manually write code, we reason about the code carefully with a purpose. Yes we do make mistakes, but the mistakes are grounded in a certain range. In contrast, AI…

One answer, as another person pointed out, is that LLM mistakes are just different. They are less explicable, less predictable, and therefore harder to spot. I can easily anticipate how an inexperienced engineer is going to mess up their first pull request for my project. I have no idea what an LLM might do. Worse, I know it might ace the first fifty pull requests and then make an absolutely mind-boggling mistake in the 51st one.

But another answer is that human autonomy is coupled to responsibility. For most line employees, if they mess up badly enough, it's first and foremost their problem. They're getting a bad performance review, getting fired, end up in court or even in prison. Because you bear responsibility for your actions, your boss doesn't have to watch what you're up to 24x7. Their career is typically not on the line unless they're deeply complicit in your misbehavior.

LLMs have no meaningful responsibility, so whoever is operating them is ultimately on the hook for what they do. It's a different dynamic. It's probably why most software engineers are not gonna get replaced by robots - your director or VP doesn't want to be liable for an agent that goes haywire - but it's also why the "oh, I have an army of 50 YOLO agents do the work while I'm browsing Reddit" is probably not a wise strategy for line employees.

Re: Vibe coding and agentic engineering are getting closer than I'd like

#353

Post by sanderjd »

jkaptur wrote: post #161
datsci_est_2015 wrote: post #71

Earlier quoted context omitted.

Bad engineers continue being bad, good engineers continue being good. I personally don’t know any colleagues who were good engineers just because they wrote code faster. The best engineers I know were ones who drew on experience and careful consideration and shared critical insights with their team that steered the direction of the system positively. > Claude, engineer a system for me, but do it good. Thanks!

> I personally don’t know any colleagues who were good engineers just because they wrote code faster. However, the best engineers I know are usually among the quickest to open an editor or debugger and use it fluently to try something out. It's precisely that speed that enables a process like "let's try X, hmm, how about Y, no... ok, Z is nice; ok team, here are the tradeoffs...". Then they remember their experience…

I really don't relate to this...

The way you apply the expert loop is to be the expert. "Can we try this...", "have you checked that...", "but what about...".

To some degree you can try to get agents to work like this themselves, but it's also totally fine (good, actually) to be nudging the work actively.

Re: Vibe coding and agentic engineering are getting closer than I'd like

#354

Post by gxs »

I grew up on construction sites with my dad. If i've done well in my career, it was from watching him operate - managing huge construction crews, how he figured out who to put on what tasks, handling suprises, setbacks, all that stuff

My dad (now retired) was always super practical about stuff. He'd tell me pretty nonchalantly things like "yeah we're dealing with xyz constraint, we may have to cut a corner over here, but that's ok", when I asked him about it he gave me a little spiel that you can be thoughtful about how you do things, including when you can cut a corner and more importantly, what corners are ok to cut.

I really took that to heart - especially the "be thoughtful about the corners you cut"

If an LLM has consistently one shotted certain tasks and they are rote/mechanical - not reviewing that code is probably ok.

Are you getting lazy and not reviewing stuff that should be reviewed even if a human wrote it? That's probably not ok

I can live with some basic code that broke because it used outdated syntax somewhere (provided the code isn't part of a mission critical application), but I can't live with it fucking JWT signing etc

Re: Vibe coding and agentic engineering are getting closer than I'd like

#355

Post by sanderjd »

zarzavat wrote: post #24

Perhaps I've missed a few weeks worth of progress, but I don't think that AIs have become more trustworthy, the errors are just more subtle. If the code doesn't compile, that's easy to spot. If the code compiles but doesn't work, that's still somewhat easy to spot. If the code compiles and works, but it does the wrong thing in some edge case, or has a security vulnerability, or introduces tech debt or dubious archite…

Yeah I relate to this. I think working in smaller chunks helps a lot. (Just like how it is for work done by humans!)

Re: Vibe coding and agentic engineering are getting closer than I'd like

#357

Post by ori_b »

truncate wrote: post #309
datsci_est_2015 wrote: post #71

Earlier quoted context omitted.

Bad engineers continue being bad, good engineers continue being good. I personally don’t know any colleagues who were good engineers just because they wrote code faster. The best engineers I know were ones who drew on experience and careful consideration and shared critical insights with their team that steered the direction of the system positively. > Claude, engineer a system for me, but do it good. Thanks!

>> Bad engineers continue being bad, good engineers continue being good. I don't know if good engineers can necessarily continue to be good. There is limit to how much careful consideration one can give if everything is on an accelerated timeline. Regardless good or not, there is limit on how much influence you have on setting those timelines. The whole playing field is changing.

It's deeper. We used to mock architects that stepped back and stopped coding, because they generated trash.

There's a cycle that is needed for good system design. Start with a problem and an approach, and write some code. As you write the code, you reify the design and flesh out the edge cases, learning where you got the details wrong. As you learn the details, you go back to the drawing board and shuffle the puzzle pieces, and try again.

Polished, effective systems don't just fall out of an engineers head. They're learned as you shape them.

Good engineers won't continue to be good when vibe-coding, because the thing that made them good was the learning loop. They may be able to coast for a while, at best.

Re: Vibe coding and agentic engineering are getting closer than I'd like

#358

Post by tbrownaw »

etothet wrote: post #33

Vibe Coding (and LLMs) did not create undisciplined engineering organizations or engineers. They exposed and accelerated them. Plenty of engineers have loose (or no!) standards and practices over how they write coee. Similarly, plenty of engineering teams have weak and loose standards over how code gets pushed to production. This concept isn't new, it's just a lot easier for individuals and teams who have never reall…

> Vibe Coding (and LLMs) did not create undisciplined engineering organizations or engineers.

Loss of discipline can be a result of panic or greed.

Perhaps believing that your own costs or your competitors' costs are suddenly becoming 10x lower could inspire one of those conditions?

(Also for greenfield projects specifically, it can plausibly be an experiment just to verify what happens. Some orgs are big enough that of course they can put a couple people on a couple-month project that'll quite likely fall flat.)

Re: Vibe coding and agentic engineering are getting closer than I'd like

#359

Post by gck1 »

mleo wrote: post #332
bitexploder wrote: post #120

Earlier quoted context omitted.

Vibe coded apps with barely no tests, invariants, etc. No wonder it turns into spaghetti. You can always refactor code, force agents to write small modular pieces and files. Good engineering is good engineering whether an agent or human wrote the code. Take time to force agents to refactor, explore choices. Humans must at least understand and drive architecture at this point still. Agents can help and do recon amazin…

I can’t understand this. The first thing I do with new agent driven project is set up quality checks. Linters, test frameworks, static analysis, etc… Whatever I would expect a developer to do, I would expect an agent to do. All implementation has to go through build success and mixed agent reviews before moving on. I might not do this with initial research/throwaway prototype, but once I know what direction to go and…

> The first thing I do with new agent driven project is set up quality checks. Linters, test frameworks, static analysis, etc

I do this too, but then I sit and observe how agent gets very creative by going around all of these layers just to get to the finish line faster.

Say, for example, if I needlessly pass a mutable reference and the linter screams at me, I know it's either linter is wrong in this case, or I should listen to it and change the signature. If I make the lazy choice, I will be dissatisfied with myself, I might even get scolded, or even fired if I keep making lazy choices.

LLM doesn't get these feelings.

LLM will almost always go for silencing it because it prevents it from reaching the 'reward'. If you put guardrails so that LLM isn't allowed to silence anything, then you get things like 'ok, I'll just do foo.accessed = 1 to satisfy the linter'.

Same story with tests. Who decides when it's the test that should be changed/deleted or the implementation?

Re: Vibe coding and agentic engineering are getting closer than I'd like

#360

Post by teddyh »

andai wrote: post #333
truncate wrote: post #309

Earlier quoted context omitted.

>> Bad engineers continue being bad, good engineers continue being good. I don't know if good engineers can necessarily continue to be good. There is limit to how much careful consideration one can give if everything is on an accelerated timeline. Regardless good or not, there is limit on how much influence you have on setting those timelines. The whole playing field is changing.

An old comic I like: - I've taken a controversial new pill that accelerates my brain. -- So you're smart now? - I'm stupid faster! That being said, being stupid faster can work if validation is cheap (and exists in the first place). Turns out "eh close enough" for AGI is just stupidity in an "until done" loop. (Technically referred to as Ralphing.)

You mean this one: https://knowyourmeme.com/photos/1567852-shen-comix>
Post reply on HN