Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vibe engineering

simonwillison.net

Post reply on HN
Print view View original

381–390 of 759 posts

Re: Vibe engineering

#381

Post by CuriouslyC »

svieira wrote: post #135
CuriouslyC wrote: post #45

Earlier quoted context omitted.

Except that we're not going to be "coding" very soon. We're going to be firing off jobs that get tracked in VCS, gated through CI, then reviewed by a panel of agents with different specialties. At the end you'll have a few select sections of code that these agents flagged for human review, and thousands of lines of stuff that you don't need to worry about.

Then it turns out that your CI gates are green because your tests are all subtly broken, the code you've been reviewing doesn't matter and the code you haven't been reviewing is what's broken in production. You learn from that and rebuild the universe, but now you're compute-limited from rebuilding your agents to deal with the underlying issue in your tooling and you have to dig yourself out of a hole you dug with a…

That's absolutely one possible outcome, just like when manually coding you have a team that doesn't know what they're doing and doesn't build anything close to what the customer wanted. YOLO'ing development has never worked, there's skill in everything.

Re: Vibe engineering

#382

Post by llamatastic »

rhetocj23 wrote: post #344
benterix wrote: post #278

Earlier quoted context omitted.

My lessons so far: 1. Less fun. 2. A lot of more "review fatigue". 3. Tons of excess code I'd never put in there in the first place. 4. Frustration with agents being too optimistic which with time verges on the ludicurous ("Task #3 has been completed successfully with 98% tests failing. [:useless_emojis:]") 5. Frustration with agents routinely getting down a rabbit hole or going in circles, the effort needed to get t…

One thing I don’t get - If you spend much of your time reviewing, you’re just reading - you’re not actually doing anything - you’re passive in the activity of code production. By extension you will become worse at knowing what a good standard of code is and become worse at reviewing code. I’m not a SWE so I have no interests to protect by criticising what is going on.

In my DJing years I've learned that it is best to provide a hot signal and trim the volume than trying to amplify it later, because you end up amplifying noise. Max out the mixer volume and put a compressor (and a limiter after to protect the speaker set up - it will make it sound awful if hit, but it won't damage your set up and it will flag clueless bozos loud and clear) later, don't try to raise it after it leaves the origin.

It seems to me that adding noise to the process and trying to cut it out later is a self defeating proposition. Or as Deming put it, (paraphrasing) you can't QC quality into a bad process.

I can see how it seems better to "move fast and break things" but I will live and die by the opposite "move slow and fix things". There's much, much more to life than maximizing short term returns over a one dimensional naïve utilitarian take on value.

Re: Vibe engineering

#383

Post by ridruejo »

This matches our experience developing with agents. In particular, as we wanted to use multiple agents in the background to do tasks, we had to really invest in different areas so they would not go in wild directions or have to ask continually for feedback, defeating the purpose of working in the background. First, we needed to provide relevant context on how to do the task (some of it is "generic" like Svelte documentation, some of it is specific to how to write tests for our particular project), be extremely detailed and specific in the prompt about what we want and how to go about it (divide it in different well defined steps) and finally provide with specific tools via MCP (like MySQL access and installing system packages). Once we consistently do all this work upfront, it really pays off because you can launch a large number of agents in parallel that don't interfere with each other (git worktrees, containerized environments) and don't require babysitting for the most part. We just open sourced the tooling we used internally for this: https://github.com/endorhq/rover

Re: Vibe engineering

#384

Post by mejutoco »

tkgally wrote: post #244

Around the time GPT-4 was released in early 2023, a similar issue arose with another profession: translation. It was at that point that machine translation between languages like English and Japanese (the language pair I have worked with) started to approach human level for the first time. I took part in a lot of discussions then with other professional translators, and the reaction of many was similar to that of som…

Translation is great to discuss LLMs. Thanks for sharing your experience.

On one side translation is not very valued by most people. It is rare that people know who translated a book, for example. It is a pity but people do not read much these days.

Additionally, or maybe because of the above, translation is often paid in terms of number of lines or words, even before AI. A bit like software security, it is often sadly just a check at the end.

IMHO the only future-proof translation fields are legal (because a human can be put in prison or pay a fine) or live translation/interpretation (because a human can go in front of people, meet them at an event, etc.).

Re: Vibe engineering

#385

Post by imiric »

guitarro wrote: post #374

I don't think the issue is with the "code" part, but more with the "vibe" part. The "vibe" part indicates that's more of a "let's just see what happens" kind of approach, which I don't think is true for people using AI generated coding in their professional coding jobs, who very much know what they want to get out of AI coding tools, how to ask for it, and how to assess the quality of the outcome. Maybe something lik…

Right.

The more accurate phrase would be "tool" or "machine-assisted development".

We don't need new terminology to describe something that has always existed because the tools we use have (arguably) improved.

"Vibe coding" is also a misnomer. People who engage in that activity are not "coding". They're using a tool to generate software until they're satisfied with the result. That has also been possible with low-code, no-code, website builders, and other tools. The more accurate term for that would be "machine-driven development".

Re: Vibe engineering

#386

Post by benterix »

executesorder66 wrote: post #368
rhetocj23 wrote: post #344

Earlier quoted context omitted.

One thing I don’t get - If you spend much of your time reviewing, you’re just reading - you’re not actually doing anything - you’re passive in the activity of code production. By extension you will become worse at knowing what a good standard of code is and become worse at reviewing code. I’m not a SWE so I have no interests to protect by criticising what is going on.

Tell that to Linus Torvalds. His whole job is just doing code review, and I'd argue he's better at coding now than he ever was before.

I'd be careful with extrapolating based on the creator of Linux and Git. His life and activities are not in line with those of more typical programmers.

Re: Vibe engineering

#387

Post by mrzool »

visarga wrote: post #379

Is "vibe engineering" a correct term for this? It's not vibe based when you scaffold constraints around the agent: automated testing, planning in advance, comprehensive documentation, automated formatting and linting, and manual QA. Don't get me wrong, I started vibe coding after reading Karpathy's post. I got the memo - don't review every line of code, don't stop it when it stumbles, let it recover on its own, trust…

> Is "vibe engineering" a correct term for this?

Anything with the word “vibe” in it sounds silly and unserious imho. What’s wrong with something neutral and descriptive like “LLM-assisted programming”? Not catchy enough?

Re: Vibe engineering

#388

Post by mejutoco »

sauercrowd wrote: post #376

I really don't think we're doing the tools or the industry any favors/justice by prefixing new terms with `vibe`. Looking at vibe coding: it suggests you're coding but you only vaguely know what's going on, so the work is the same (coding) but the outcome may or may not be what you want Why dont we flip it around? We want a term that suggests that a fixed amount of work (coding) to be more efficient/leveraged. So why…

Some more options (I used AI with a nuanced not-too-lazy prompt, I hope it is okay):

Prompt Engineering / Directed Prompting

Architectural Steering

Spec-Driven Development

Intent-Based Coding

Critical Synthesis

Iterative Refinement

Guided Iteration

Hypothesis-Driven Coding

AI-Assisted Engineering

Cognitive Pair Programming

Dialogic Programming

Structured Prompting for Code (SPC)

Code Shepherding

AI-Assisted Engineering is probably the most descriptive. My favourite are Critical Synthesis and Code Shepherding. Both abbreviate to CS

Re: Vibe engineering

#389

Post by sgt101 »

bonoboTP wrote: post #350
sally_glance wrote: post #262

Earlier quoted context omitted.

Well said! Reading this I feel reminded of the early protests against industrialization and automation in other fields. Checks all the same boxes - insecurity and fear about the future, alienation towards the new tools, ... Not saying AI is similar in impact to the loom or something, it just occured to me how close this is to early Luddite texts.

The OG alienator was the Agricultural Revolution, settling and toiling repetitively in predetermined ways, unlike the more adventurous lifestyle from before with all the camping, hunting, gathering, where circumstances brought always novel challenges, you could be a man spearing a deer, instead of just killing some docile domesticated cow. Searching for pheasant eggs and being happy if you found some, instead of goin…

Although, gradually, all over the world people chose that lifestyle rather than take their chances with the seasons and the hunt.

Re: Vibe engineering

#390

Post by nkrisc »

executesorder66 wrote: post #368
rhetocj23 wrote: post #344

Earlier quoted context omitted.

One thing I don’t get - If you spend much of your time reviewing, you’re just reading - you’re not actually doing anything - you’re passive in the activity of code production. By extension you will become worse at knowing what a good standard of code is and become worse at reviewing code. I’m not a SWE so I have no interests to protect by criticising what is going on.

Tell that to Linus Torvalds. His whole job is just doing code review, and I'd argue he's better at coding now than he ever was before.

Linus Torvalds is hardly typical.
Post reply on HN