Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

maggieappleton.com

Post reply on HN
Print view View original

161–170 of 466 posts

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#161

Post by joshstrange »

kaydub wrote: post #61
jrmg wrote: post #27

Earlier quoted context omitted.

This is also my experience. Everything I’ve ever tried to vibe code has ended up with off-by-one errors, logic errors, repeated instances of incorrect assumptions etc. Sometimes they appear to work at first, but, still, they have errors like this in them that are often immediately obvious on code review and would definitely show up in anything more than very light real world use. They _can_ usually be manually tidied…

Yeah, it sounds like "you're holding it wrong" Like, why are you manually tidying and fixing things? The first pass is never perfect. Maybe the functionality is there but the code is spaghetti or untestable. Have another agent review and feed that review back into the original agent that built out the code. Keep iterating like that. My usual workflow: Agent 1 - Build feature Agent 2 - Review these parts of the code,…

I'm not trying to be rude here at all but are you manually verifying any of that? When I've had LLMs write unit tests they are quick to write pointless unit tests that seem impressive "2123/2123 tests passed!" but in reality it's testing mostly nothing of value. And that's when they aren't bypassing commit checks or just commenting out tests or saying "I fixed it all" while multiple tests are broken.

Maybe I need a stricter harness but I feel like I did try that and still didn't get good results.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#162

Post by m4rtink »

fragmede wrote: post #111
crote wrote: post #33

Earlier quoted context omitted.

The big difference is that compilation is deterministic: compile the same program twice and it'll generate the same output twice. It also doesn't involve any "creativity": a compiler is mostly translating a high-level concept into its predefined lower-level components. I don't know exactly what my code compiles to, but I can be pretty certain what the general idea of the assembly is going to be. With LLMs all bets ar…

> The big difference is that compilation is deterministic: compile the same program twice and it'll generate the same output twice. That's barely true now. Nix comes close, but builds are only bit-for-bit identical if you set a bunch of extra flags that aren't set by default. The most obvious instability is CPU dispatch order (aka modern single computer systems are themselves distributed, racy systems) changes the ge…

If your builds are not deterministic for the same set of inputs, you are doing something wrong - or victim of supply chain attack.

https://reproducible-builds.org/

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#163

Post by WesolyKubeczek »

causalmodels wrote: post #130
WesolyKubeczek wrote: post #92

Earlier quoted context omitted.

> Ok but this entire idea is very new. Its not an honest criticism to say no one has tried the new idea when they are actively doing it. Not really new. Back in the day companies used to outsource their stuff to the lowest bidder agencies in proverbial Elbonia, never looked at the code, and then panickedly hired another agency when the things visibly were not what was ordered. Case studies are abound on TheDailyWTF f…

Fair point on the Elbonia comparison. But we can't sue the SQLite maintainers either, and yet we trust them with basically everything. The reason is that open source developed its own trust mechanisms over decades. We don't have anything close to that with LLMs today. What those mechanisms might look like is an open question that is getting more important as AI generated code becomes more common.

> But we can't sue the SQLite maintainers either, and yet we trust them with basically everything.

But you don’t pay them any money and don’t enter into contractual relationship with them either. Thus you can’t sue them. Well, you can try, of course, but.

You could sue an Elbonian company, though, for contract breach. LLMs are like usual Elbonian quality with two middlemen but quicker, and you only have yourself to blame when they inevitably produce a disaster.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#164

Post by zerkten »

jplusequalt wrote: post #9
bbayles wrote: post #8

Earlier quoted context omitted.

I'm sympathetic to this view, but I also wonder if this is the same thing that assembly language programmers said about compilers. What do you mean that you never look at the machine code? What if the compiler does something inefficient?

>but I also wonder if this is the same thing that assembly language programmers said about compilers But as a programmer writing C code, you're still building out the software by hand. You're having to read and write a slightly higher level encoding of the software. With vibe coding, you don't even deal with encodings. You just prompt and move on.

I've wondered if people who write detailed specs, are overly detailed, are in a regulated industry, or even work with offshore teams have success more quickly simply they start with that behavior. Maybe they have a tendency to dwell before moving on which may be slightly more iterative than someone who vibecodes straight through.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#165

Post by notpachet »

luckydata wrote: post #147
alecbz wrote: post #80

Earlier quoted context omitted.

Even ignoring determinism, with traditional source code you have a durable, human-readable blueprint of what the software is meant to do that other humans can understand and tweak. There's no analogy in the case of "don't read the code" LLM usage. No artifacts exist that humans can read or verify to understand what the software is supposed to be doing.

yeah there is. it's called "documentation" and "requirements". And it's not like you can't go read the code if you want to understand how it works, it's just not necessary to do so while in the process of getting to working software. I truly do not understand why so many people are hung up on this "I need to understand every single line of code in my program" bs I keep reading here, do you also disassemble every libr…

> do you also disassemble every library you use and understand it?

Sometimes.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#166

Post by pstuart »

an0malous wrote: post #124
hyperpape wrote: post #97

Earlier quoted context omitted.

> If you read Steve's writeup, it's clear that this is a big fun experiment: So, Steve has the big scary "YOU WILL DIE" statements in there, but he also has this: > I went ahead and built what’s next. First I predicted it, back in March, in Revenge of the Junior Developer. I predicted someone would lash the Claude Code camels together into chariots, and that is exactly what I’ve done with Gas Town. I’ve tamed them to…

I agree, I’m one of the Very Serious Engineers and I liked Steve’s post when I thought it was sort of tongue in cheek but was horrified to come to the HN comments and LinkedIn comments proclaiming Gastown as the future of engineering. There absolutely is a large contingent of engineers who believe this, and it has a real world impact on my job if my bosses think you can just throw a dozen AI agents at our product roa…

AI is such a fun topic -- the hype makes it easy to loath, but as a coder working with Claude I think it's an awesome tool.

Gastown looks like a viable avenue for some app development. One of the most interesting things I've noticed about AI development is that it forces one to articulate desired and prohibited behaviors -- a spec becomes a true driving force.

Yegge's posts are always hyperbolic and he consistently presents interesting takes on the industry so I'm willing to cut him a buttload of slack.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#167

Post by lowbloodsugar »

an0malous wrote: post #124
hyperpape wrote: post #97

Earlier quoted context omitted.

> If you read Steve's writeup, it's clear that this is a big fun experiment: So, Steve has the big scary "YOU WILL DIE" statements in there, but he also has this: > I went ahead and built what’s next. First I predicted it, back in March, in Revenge of the Junior Developer. I predicted someone would lash the Claude Code camels together into chariots, and that is exactly what I’ve done with Gas Town. I’ve tamed them to…

I agree, I’m one of the Very Serious Engineers and I liked Steve’s post when I thought it was sort of tongue in cheek but was horrified to come to the HN comments and LinkedIn comments proclaiming Gastown as the future of engineering. There absolutely is a large contingent of engineers who believe this, and it has a real world impact on my job if my bosses think you can just throw a dozen AI agents at our product roa…

I too am a Very Serious Engineer but my shock is in the other direction: of course the ideas behind Gas Town are the future of software development and several VSEs I know are developing a proper, robust, engineering version of it that works. As the author of this article here remarks “yes, but Steve did it first”, and it annoys me that if I had written this post nobody would have read it, but also that, because I intend to use it in Very Serious Business ($bns) my version isn’t ready to a actually be published yet. Bravo to Steve for getting these thoughts on paper and the idea built even in such crude form. But “level 8” is real and there will be 9s and 10s and I am really enjoying building my own.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#168

Post by conartist6 »

luckydata wrote: post #151
conartist6 wrote: post #29

Earlier quoted context omitted.

I write JS, and I have never directly observed the IRs or assembly code that my code becomes. Yet I certainly assume that the compiler author has looked at the compiled output in the process of writing a compiler! For me the difference is prognosis. Gas Town has no ratchet of quality: its fate was written on the wall since the day Steve decided he didn't want to know what the code says: it will grow to a moderate but…

or he will find a way to get the AI to create harnesses so it becomes stable. The lack of imagination and willingness to experiment in the HN crowd is AMAZING me and worrying me at the same time. Never thought a group of engineers would be the most conservative and close minded people I could discuss with.

It's a paradox, huh. If the AI harness became so stable it wrote good code he wouldn't be afraid to look at the code he would be eager to look at it, right? But then if it mattered if AI wrote good code or not he couldn't defend his position that the way to create value with code is quantity over quality. He needs to sell the idea of something only AI can do, which means he needs the system to be made up of a lot of bad or low quality code which no person would ever want to be forced to look at.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#169

Post by dceddia »

joshstrange wrote: post #159
suriya-ganesh wrote: post #7

>Yegge is leaning into the true definition of vibecoding with this project: “It is 100% vibecoded. I’ve never seen the code, and I never care to.” I don't get it. Even with a very good understanding of what type of work I am doing and a prebuilt knowledge of the code, even for very well specced problem. Claude code etc. just plain fail or use sloppy code. How do these industry figures claim they see no part of a 225K…

Where is the "super upvote button" when you need it? YES! I have been playing with vibe coding tools since they came out. "Playing" because only on rare occasions have I created something that is good enough to commit/keep/use. I keep playing with them because, well I have a subscription, but also so I don't fall into the fuddy-duddy camp of "all AI is bad" and can legitimately speak on the value, or lack thereof, of…

So I’m probably in a similar spot - I mostly prompt-and-check, unless it’s a throwaway script or something, and even then I give it a quick glance.

One thing that stands out in your steps and that I’ve noticed myself- yeah, by prompt 10, it starts to suck. If it ever hits “compaction” then that’s beyond the point of return.

I still find myself slipping into this trap sometimes because I’m just in the flow of getting good results (until it nosedives), but the better strategy is to do a small unit of work per session. It keeps the context small and that keeps the model smarter.

“Ralph” is one way to do this. (decent intro here: https://www.aihero.dev/getting-started-with-ralph)

Another way is “Write out what we did to PROGRESS.md” - then start new session - then “Read @PROGRESS.md and do X”

Just playing around with ways to split up the work into smaller tasks basically, and crucially, not doing all of those small tasks in one long chat.

Re: Gas Town's agent patterns, design bottlenecks, and vibecoding at scale

#170

Post by alecbz »

luckydata wrote: post #147
alecbz wrote: post #80

Earlier quoted context omitted.

Even ignoring determinism, with traditional source code you have a durable, human-readable blueprint of what the software is meant to do that other humans can understand and tweak. There's no analogy in the case of "don't read the code" LLM usage. No artifacts exist that humans can read or verify to understand what the software is supposed to be doing.

yeah there is. it's called "documentation" and "requirements". And it's not like you can't go read the code if you want to understand how it works, it's just not necessary to do so while in the process of getting to working software. I truly do not understand why so many people are hung up on this "I need to understand every single line of code in my program" bs I keep reading here, do you also disassemble every libr…

> it's called "documentation" and "requirements"

What I mean is an artifact that is the starting point for generating the software. Compiled binaries can be completely thrown away whenever because you know you have a blueprint (the source code) that can reliably reproduce it.

Documentation & requirements _could_ work this way if they served as input to the LLMs that would then go and create the source code from scratch. I don't think many people are using LLMs this way, but I think this is an interesting idea. Maybe soon we'll have a new generation of "LLM-facing programming languages" that are even higher level software blueprints that will be fed to LLMs to generate code.

TDD is also a potential answer here? You can imagine a world where humans just write test suites and LLMs fill out the code to get it to pass. I'm curious if people are using LLMs this way, but from what I can tell a lot of people use them for writing their tests as well.

> And it's not like you can't go read the code if you want to understand how it works

In-theory sure, but this is true of assembly in-theory as well. But the assembly of most modern software is de-facto unreadable, and LLM-generated source code will start going that way too the more people become okay with not reading it. (But again, the difference is that we're not necessarily replacing it with some higher-level blueprint that humans manage, we're just relying on the LLMs to be able to manage it completely)

> I truly do not understand why so many people are hung up on this "I need to understand every single line of code in my program" bs I keep reading here, do you also disassemble every library you use and understand it? no, you just use it because it's faster that way.

I think at the end of the day this is just an empirical question: are LLMs good enough to manage complex software "on their own", without a human necessarily being able to inspect, validate, or help debug it? If the answer is yes, maybe this is fine, but based on my experiences with LLMs so far I am not convinced that this is going to be true any time soon.

Post reply on HN