Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Perverse incentives of vibe coding

fredbenenson.medium.com

Post reply on HN
Print view View original

191–200 of 239 posts

Re: Perverse incentives of vibe coding

#191

Post by tauoverpi »

motorest wrote: post #190
tauoverpi wrote: post #185

Earlier quoted context omitted.

I've had the same issue every time I've tried it. The code I generally work on is embedded C/C++ with in-house libraries where the tools are less than useful as they try to generate non-existant interfaces and generally generate worse code than I'd write by hand. There's a need for correctness and being able to explain the code thus use of those tools is also detrimental to explainability unless I hand-hold it to the…

> I've had the same issue every time I've tried it. The code I generally work on is embedded C/C++ with in-house libraries where the tools are less than useful as they try to generate non-existant interfaces and generally generate worse code than I'd write by hand. That's because whatever training the model had, it didn't covered anything remotely similar to the codebase you worked on. We get this issue even with obs…

I'm aware of the _why_ but this is why the tools aren't useful for my case. If they cannot consume the codebase in a reasonable amount of time and provide value from that then they generally aren't useful in areas where I would want to use them (navigating large codebases). If the codebase is relatively small or the problem is known then an LLM is not any better than tab-complete and arguably worse in many cases as the generated result has to be parsed and added to my mental model of the problem rather than the mental model being constructed while working on the code itself.

I guess my point is, I have no use for LLMs in their current state.

> That's because whatever training the model had, it didn't covered anything remotely similar to the codebase you worked on. > We get this issue even with obscure FLOSS libraries.

This is the issue however as unfamiliar codebases is exactly where I'd want to use such tooling. Not working in those cases makes it less than useful.

> Unless you provide them with context or instruct them not to make up stuff, they will resort to bullshit their way into an example.

In all cases context was provided extensively but at some point it's easier to just write the code directly. The context is in surrounding code which if the tool cannot pick up on that when combined with direction is again less than useful.

> What's truly impressive about this is that often times the hallucinated code actually works.

I haven't experienced the same. It fails more often than not and the result is much worse than the hand-written solution regardless of the level of direction. This may be due to unfamiliar code but again, if code is common then I'm likely familiar with it already thus lowering the value of the tool.

> Again,this suggest a failure on your side for not providing any context.

This feels like a case of blaming the user without full context of the situation. There are comments, the names are descriptive and within reason, and there's annotation of why certain things are done the way they are. The purpose of a doc comment is not "this does X" but rather _why_ you want to use this function and it's purpose which is something LLMs struggle to derive from my testing of them. Adding enough direction to describe such is effectively writing the documentation with a crude english->english compiler between. This is the same problem with unit test generation where unit tests are not to game code coverage but to provide meaningful tests of the domain and known edge cases of a function which is again something the LLM struggles with.

For any non-junior task LLM tools are practically useless (from what I've tested) and for junior level tasks it would be better to train someone to do better.

Re: Perverse incentives of vibe coding

#192

Post by flashgordon »

This addiction and fear of things-going-bad-if-i-dont-listen-to-the-copilot is precisely why my workflow is a bit more simple and caveman-ish:

1. start a project with vague README (or take an existing one).

2. create makefile with the "prompt" action that looks something like (I might put it in a script to work around tabs etc):

```

prompt:

    for f in `find ./ | grep '*.go *.ts *.files_i_care_about' | grep -v 'files to ignore' | pbcopy`

    do

        echo "// FILE: $f"

        cat $f

    done
```

3. Run `make prompt` to get a fresh new starting prompt, Go to Gemini (AI Studio) and use the prompt:

```You have the following files. Understand it and we will start building some features.

```

4. It thinks, understands and gives me the "I am ready" line.

5. To build feature X I simply prompt it with:

``` I want to build feature X. Understand it, plan it, and do not regenerate entire files. Just give me unix style diffs. ```

6. Iterate on what i like and dont (including refactors, etc)

7. Copy patches and apply locally

8. Repeat steps 5 - 7.

10. After about 300-400k tokens generated (say over 20-40 features) I snapshot with the prompt:

``` Great now is a great time to checkpoint. Generate a SUMMARY.md on a per folder basis of your understanding of the current state of the project along with a roadmap of next steps. ```

11. I save/update the SUMMARY.md and go to bed. When I come back I repeat from step 2 - and voila the SUMMARY.md generated before are included too.

I have generated about 20M tokens so far at a cost of 0. For me "copy/pasting" diffs is not a big deal. Getting clean code, having a nice custom workflow is more important. I am still ready to relinquish control fully to an agent. I just want a really good code search/auto-complete out of the LLM that adheres to *my* interfaces and constraints.

Re: Perverse incentives of vibe coding

#193

Post by bitwize »

"Vibe coding as gacha game" is a new wrinkle I didn't expect. It certainly explains why I see people who should know better talking up AI and LLMs like they're the second coming: it's like how stoners talk about weed as a cancer cure.

Re: Perverse incentives of vibe coding

#194

Post by nbittich »

At best, the only useful thing I can get from chat gpt, deepSeek, or grok is keywords I can search on Google to find a valid solution to my problem. I get so frustrated with them that I almost never use LLMs, except for fixing grammar or translating. It's not because I'm against them, but because they are useless to me and a massive waste of time.

Re: Perverse incentives of vibe coding

#195

Post by theshrike79 »

lexandstuff wrote: post #157
tptacek wrote: post #127

Earlier quoted context omitted.

The first and most important question to ask here is: are you using a coding agent? A lot of times, people who aren't getting much out of LLM-assisted coding are just asking Claude or GPT for code snippets, and pasting and building them themselves (or, equivalently, they're using LLM-augmented autocomplete in their editor). Almost everybody doing serious work with LLMs is using an agent, which means that the LLM is a…

Funny, I would give the absolute opposite advice. In my experience, the use of agents (mainly Cursor) is a sure-fire way to have a really painful experience with LLM-assisted coding. I much prefer to use AI as a pair programmer, that I talk to and sometimes get to write entire files, but I'm always the one doing the driving, and mostly the one writing the code. If you aren't building up mental models of the problem a…

Agents make it easier for you to give context to the LLM, or for it to grab some by itself like Cline/Claude/Cursor/Windsurf can do.

With a web-based system you need repomix or something similar to give the whole project (or parts of it if you can be bothered to filter) as context, which isn't exactly nifty

Re: Perverse incentives of vibe coding

#196

Post by theshrike79 »

__mharrison__ wrote: post #133
tptacek wrote: post #127

Earlier quoted context omitted.

The first and most important question to ask here is: are you using a coding agent? A lot of times, people who aren't getting much out of LLM-assisted coding are just asking Claude or GPT for code snippets, and pasting and building them themselves (or, equivalently, they're using LLM-augmented autocomplete in their editor). Almost everybody doing serious work with LLMs is using an agent, which means that the LLM is a…

What agent do you recommend?

I've settled on Cline for now, with openrouter as the backend for LLMs, Gemini 2.5 for planning and Claude 3.7 for act mode.

Cursor is fine, Claude Code and Aider are a bit too janky for me - and tend to go overboard (making full-ass git commits without prompting) and I can't be arsed to rein them in.

Re: Perverse incentives of vibe coding

#197

Post by theshrike79 »

brooke2k wrote: post #98

I don't understand the productivity that people get out of these AI tools. I've tried it and I just can't get anything remotely worthwhile unless it's something very simple or something completely new being built from the ground up. Like sure, I can ask claude to give me the barebones of a web service that does some simple task. Or a webpage with some information on it. But any time I've tried to get AI services to h…

Just yesterday I wanted to adjust the Renovate setup in a repo.

I could spend 5-10 minutes digging on through the docs for the correct config option, or I can just tap a hotkey, open up GitHub Copilot in Rider and tell it what I want to achieve.

And within seconds it had a correct-looking setting ready to insert to my renovate.json file. I added it, tested it and it works.

I kinda think people who diss AIs are prompting something like "build me Facebook" and then being disappointed when it doesn't :D

Re: Perverse incentives of vibe coding

#198

Post by theshrike79 »

andy99 wrote: post #29

I wish more had been written about the first assertion that using an LLM to code is like gambling and you're always hoping that just one more prompt will get you what you want. It really captures how little control one has over the process, while simultaneously having the illusion of control. I don't really believe that code is being made verbose to make more profits. There's probably some element of model providers…

But just like gambling, there are ways to do it correctly.

Yes, there are the grandmas in a trance vibe-gambling by shoving a bucket of quarters in a slot machine.

But you also have people playing Blackjack and beating the averages by knowing how it's played, maybe having a "feel" for the deck (or counting cards...), and most importantly knowing when to fold and walk away.

Same with LLMs, you need to understand context sizes and prompts and you need to have a feel for when the model is just chasing its own tail or trying to force a "solution" just to please the user.

Re: Perverse incentives of vibe coding

#199

Post by overfeed »

tptacek wrote: post #164
lexandstuff wrote: post #157

Earlier quoted context omitted.

Funny, I would give the absolute opposite advice. In my experience, the use of agents (mainly Cursor) is a sure-fire way to have a really painful experience with LLM-assisted coding. I much prefer to use AI as a pair programmer, that I talk to and sometimes get to write entire files, but I'm always the one doing the driving, and mostly the one writing the code. If you aren't building up mental models of the problem a…

I mean, it might depend, but many of the most common complaints about LLM coding (most notably hallucination) are essentially solved problems if you're using agents. Whatever works for you! I don't even like autocomplete, so I sympathize with not liking agents. If it helps, for context: I'll go round and round with an agent until I've got roughly what I want, and then I go through and beat everything into my own idio…

> ...many of the most common complaints about LLM coding (most notably hallucination) are essentially solved problems if you're using agents

Inconsistency and crap code quality aren't solved yet, and these make the agent workflow worse because the human only gets to nudge the AI in the right direction very late in the game. The alternative, interactive, non-agentic workflows allow for more AI-hand-holding early, and better code quality, IMO.

Agents are fine if no human is going to work on the (sub)system going forward, and you only care about the shiny exterior without opening the hood to witness the horrors within.

Re: Perverse incentives of vibe coding

#200

Post by jiggawatts »

tptacek wrote: post #130
jiggawatts wrote: post #118

Earlier quoted context omitted.

My favourite example of this is the recent trend towards “wide events” replacing logs and metrics… spearheaded and popularised by companies that charge by the gigabytes ingested.

Companies that ingest logs generally rip their customers faces off with their pricing. At least oTel spans can be tail-sampled.

I worked out that it's cheaper to write logs to high-end Samsung SSDs and then throw them away every month than to retain them in the log analytics systems of some cloud services for the same period of time.

Wait, no, sorry... that doesn't quite "paint the right picture".

The "single use" SSDs are 75 times cheaper than storing the data in the cloud.

Post reply on HN