Live data from Hacker News

I know when you're vibe coding

alexkondov.com

31–40 of 178 posts

Re: I know when you're vibe coding

#31
In my experience pretty much all of these issues stem from a combination of short context windows and suboptimal “context engineering”.

If the agent has a clean, relevant context explaining what global functions are available it tends to use them properly.

The biggest challenge is how to construct the right context for each request, and keep it clean until the feature is finished. I expect we will see a lot of improvements in this area the coming months (sub-agents being an obvious example).

Re: I know when you're vibe coding

#32
I personally treat the LLM as a very junior programmer. He's willing to work, will take instructions, but his knowledge of the codebase, and patterns we use is lacking strongly. So it needs a LOT of handholding, very clear instructions, description of potential pitfalls, and smaller, scoped tasks, and reviewed carefully to catch any straying off pattern.

Also, I make it work the same way I do: I first come up with the data model until it "works" in my head, before writing any "code" to deal with it. Again, clear instructions.

Oh another thing, one of my "golden rule" is that it needs to keep a block comment at the top of the file to describe what's going on in that file. It acts as a second "prompt" when I restart a session.

It works pretty well, it doesn't appear as "magic" as the "make it so!" approach people think they can get away with, but it works for me.

But yes, I still also spend maybe 30% of the time cleaning up, renaming stuff and do more general rework of the code before it comes "presentable" but it still allows to work pretty quickly, a lot quicker than if I were to do it all by hand.

Re: I know when you're vibe coding

#33
post #20

This is all true. The best way to treat LLM's as they are now is one step above the abstraction offered by compiled languages over assembly. You can describe something in plain english, note its explicit requirements, inputs and outputs, and an LLM can effectively write the code as a translation of the logic you are specifying. Using LLM's, you are best served minimizing the entropy they have to deal with. The transf…

This is a false analogy. LLMs do not "compile" natural language to high level code in the same way that a compiler or interpreter implements a high-level programming language in terms of machine instructions (or, for that matter, how a CPU implements machine instructions in hardware). Programming and machine languages aim for a precise and unambiguous semantics, such that it's meaningful to talk about things like whe…

If a program calls an API like

search_engine.get_search_results(query, length, order)

It doesn't "care" about the algorithm that produced that list of results, only that it fits the approximation of how the algorithm works as defined by the schema. There are thousands of ways the engine could have been implemented to produce the schema that returns relevance-based results from a web-crawler-sourced database.

In the same way, if I prompt an LLM "design a schema with [list of requirements] that works in [code context and API calls]", there are thousands of ways it could produce that code, but within a margin of error a high quality LLM should be able to produce the code that fits those requirements.

Of course the difference is that there is a stochastic element to LLM generated code. However it is useful to think of LLM's this way because it allows being able to leverage their probability of being correct, even if they aren't as precise as calling APIs but being explicit in how those abstractions are used.

Re: I know when you're vibe coding

#34
This matches my experience. And I do mind the unnecessary comments. I always tell juniors that every line is important and the presence of a comment is a powerful tool for communicating with future developers. Think of it like labelling circuits on a fusebox. Now these garbage code generators are putting comments on everything because they don't understand anything and can't tell what it's redundant.

But this is only scratching the surface of what's wrong, as the article elaborates.

The thing is people claim these things are making them faster. I don't believe it. What I believe is they are faster at generating shit. I know that because a baby can coax an LLM into producing shit too.

I do not believe you can spend that much time writing the correct prompt - use this exact function, follow this pattern, add a comment here, don't add one there, no, not like that - and still be quicker than just writing it yourself directly in the language.

It's like if I speak French fluently but only communicate through a translator that I instruct in English but constantly have to correct when they miss the nuance in my speech. I'd just speak French!

So, no, I don't believe it.

What I believe is that many, many software developers have been manually writing boilerplate, repetitive and boring code over and over again up until this point. I believe it because I've seen it. LLMs will obviously speed this up. But some of us already learnt how to use the computer to do that for us.

What I also believe is developers exist who don't understand, or care to understand, what they are doing. They will code using a trial and error approach and find solutions based purely on perceived behaviour of the software. I believe it because I've seen it. Of course LLMs will speed up this process. But some of us actually think about what we're writing, just like we don't just randomly string together words in a restaurant and then just keep trying until we get the dish we want.

Re: I know when you're vibe coding

#35

Earlier quoted context omitted.

>No, LLMs will not get better. What makes you so sure of this? They've been getting better like clockwork every few months for the past 5 years.

No they haven't. The hallucinate exactly as much as they did five years ago.

They really don’t though.

Re: I know when you're vibe coding

#36
post #26

> Because no one would write an HTTP fetching implementation covering all edge cases when we have a data fetching library in the project that already does that. > No one would implement a bunch of utility functions that we already have in a different module. > No one would change a global configuration when there’s a mechanism to do it on a module level. > No one would write a class when we’re using a functional appr…

This. Reinventing the wheel at every opportunity, forgetting about or ignoring the expected way to do something, mixing patterns, you name it. The author may call it "vibe coding", that's fine but it has little to do with LLMs. The tool has the same amount of care anyone rushing to get something done, or that hasn't build the project themselves, or maybe doesn't have enough experience would. I can only assume it's a not-very-subtle complaint about a specific person in their team, "written in a way no developer on the team would" is telling.

I'd be extremely careful about applying this thinking anywhere else. There's enough baseless finger-pointing in academia and arts already.

Re: I know when you're vibe coding

#37
post #8

Earlier quoted context omitted.

No, LLMs are not an "abstraction" like a compiler is. This is bullshit. LLMs are stochastic token generators. I have NEVER met someone in real life that has produced something I wouldn't throw in the trash using LLMs, and I have had the displeasure of eating cookies baked from an LLM recipe. No, LLMs will not get better. The singularity bullshit has been active since 2010s. LLMs have consumed the entire fucking Inter…

>No, LLMs will not get better. What makes you so sure of this? They've been getting better like clockwork every few months for the past 5 years.

I don't claim that they won't get better, but they certainly haven't gotten better. From the original release of ChatGPT to now, they still suck in the same exact ways.

Re: I know when you're vibe coding

#38
post #26

> Because no one would write an HTTP fetching implementation covering all edge cases when we have a data fetching library in the project that already does that. > No one would implement a bunch of utility functions that we already have in a different module. > No one would change a global configuration when there’s a mechanism to do it on a module level. > No one would write a class when we’re using a functional appr…

I have worked on such teams. Mostly, even. I would not accept any PRs with code doing any of those things (human or machine made). Small(er) teams on small to medium sized projects.

Critical solutions, but small(er) projects with 2-4 devs, that’s where it’s at. I feel like it’s because then it’s actually possible to build a devteam culture and consensus that has the wanted balance of quality and deliveryspeed.

Re: I know when you're vibe coding

#39
heck, I'm sitting in a team with a code base so full of slop that was written by humans, the AI can't even fix it and I'm burned out from trying to make it better. and I get told to be quiet about code standards because the team is still learning.

Re: I know when you're vibe coding

#40
post #31

In my experience pretty much all of these issues stem from a combination of short context windows and suboptimal “context engineering”. If the agent has a clean, relevant context explaining what global functions are available it tends to use them properly. The biggest challenge is how to construct the right context for each request, and keep it clean until the feature is finished. I expect we will see a lot of improv…

> If the agent has a clean, relevant context explaining what global functions are available it tends to use them properly.

STOP! The agent does not exist. There are no agents; only mathematical functions that have an input and produce an output.

Stop anthropomorphizing LLMs, they are not human, they don’t do anything.

It might seem like it does not matter; my take is it’s primordial. Humans are not machines and vice-versa.

Post reply on HN