Earlier quoted context omitted.
GitHub copilot was released in 2021, and Cursor was released around October 2023[0]. [0]: https://news.ycombinator.com/item?id=37888477
At the earliest, "vibecoding" was only possible with Claude 3.5, released July 2024 ... maaaybe Claude 3, released in March of that year... It's worth mentioning that even today, Copilot is an underwhelming-to-the-point-obstructing kind of product. Microsoft sent salespeople and instructors to my job, all for naught. Copilot is a great example of how product > everything, and if you don't have a good product... well.…
After two years of vibecoding, I'm back to writing by hand
21–30 of 652 posts
Re: After two years of vibecoding, I'm back to writing by hand
#22> Not only does an agent not have the ability to evolve a specification over a multi-week period as it builds out its lower components, it also makes decisions upfront that it later doesn’t deviate from. That's your job. The great thing about coding agents is that you can tell them "change of design: all API interactions need to go through a new single class that does authentication and retries and rate-limit throttl…
> (And the automated test suite will help them confirm that the refactoring worked properly, because naturally you had them construct an automated test suite when they built those original features, right?) I dunno, maybe I have high standards but I generally find that the test suites generated by LLMs are both over and under determined. Over-determined in the sense that some of the tests are focused on implementatio…
Or I can say "use pytest-httpx to mock the endpoints" and Claude knows what I mean.
Keeping an eye on the tests is important. The most common anti-pattern I see is large amounts of duplicated test setup code - which isn't a huge deal, I'm much more more tolerant of duplicated logic in tests than I am in implementation, but it's still worth pushing back on.
"Refactor those tests to use pytest.mark.parametrize" and "extract the common setup into a pytest fixture" work really well there.
Generally though the best way to get good tests out of a coding agent is to make sure it's working in a project with an existing test suite that uses good patterns. Coding agents pick the existing patterns up without needing any extra prompting at all.
I find that once a project has clean basic tests the new tests added by the agents tend to match them in quality. It's similar to how working on large projects with a team of other developers work - keeping the code clean means when people look for examples of how to write a test they'll be pointed in the right direction.
One last tip I use a lot is this:
Clone datasette/datasette-enrichments
from GitHub to /tmp and imitate the
testing patterns it uses
I do this all the time with different existing projects I've written - the quickest way to show an agent how you like something to be done is to have it look at an example.Re: After two years of vibecoding, I'm back to writing by hand
#23Re: After two years of vibecoding, I'm back to writing by hand
#24Earlier quoted context omitted.
> My way is to understand my own codebase and look at the output of the LLM. Then you are not vibe coding. The core, almost exclusive requirement for "vibe coding" is that you DON'T look at the code. Only the product outcome.
I know what you mean but to look that black and white at it seems dismissive of the spectrum that's actually there (between vibecoding and software engineering). Looking at the whole spectrum is, I find, much more interesting. Normally I'd know 100% of my codebase, now I understand 5% of it truly. The other 95% I'd need to read it more carefully before I daresay I understand it.
I agree there is a spectrum, and all the way to the left you have "vibe coding" and all the way to the right you have "manual programming without AI", of course it's fine to be somewhere in the middle, but you're not doing "vibe coding" in the way Karpathy first meant it.
Re: After two years of vibecoding, I'm back to writing by hand
#25AI is incredibly dangerous because it can do the simple things very well, which prevents new programmers from learning the simple things ("Oh, I'll just have AI generate it") which then prevents them from learning the middlin' and harder and meta things at a visceral level. I'm a CS teacher, so this is where I see a huge danger right now and I'm explicit with my students about it: you HAVE to write the code. You CAN'…
Your curriculum may be different than it is around here, but here it's frankly the same stuff I was taught 30 years ago. Except most of the actual computer science parts are gone, replaced with even more OOP, design pattern bullshit.
That being said. I have no idea how you'd actually go about teaching students CS these days, considering a lot of them will probably use ChatGPT or Claude regardless of what you do. That is what I see in the statistic for grades around here. For the first 9 years I was a well calibrated grader, but these past 1,5ish years it's usually either top marks or bottom marks with nothing in between. Which puts me outside where I should be, but it matches the statistical calibration for everyone here. I obviously only see the product of CS educations, but even though I'm old, I can imagine how many corners I would have cut myself if I had LLM's available back then. Not to mention all the distractions the internet has brought.
Re: After two years of vibecoding, I'm back to writing by hand
#26Earlier quoted context omitted.
> My way is to understand my own codebase and look at the output of the LLM. Then you are not vibe coding. The core, almost exclusive requirement for "vibe coding" is that you DON'T look at the code. Only the product outcome.
I think this is my main confusion around vibe coding. Is it a skill for the layman? Or does it only work if you have the understanding you would need to manage a team of junior devs to build a project. I feel like we need a different term for those two things.
Programming together with AI however, is a skill, mostly based on how well you can communicate (with machines or other humans) and how well your high-level software engineering skills are. You need to learn what it can and cannot do, before you can be effective with it.
Re: After two years of vibecoding, I'm back to writing by hand
#27AI is incredibly dangerous because it can do the simple things very well, which prevents new programmers from learning the simple things ("Oh, I'll just have AI generate it") which then prevents them from learning the middlin' and harder and meta things at a visceral level. I'm a CS teacher, so this is where I see a huge danger right now and I'm explicit with my students about it: you HAVE to write the code. You CAN'…
This is the ultimate problem with AI in academia. We all inherently know that “no pain no gain” is true for physical tasks, but the same is true for learning. Struggling through the new concepts is essentially the point of it, not just the end result.
Of course this becomes a different thing outside of learning, where delivering results is more important in a workplace context. But even then you still need someone who does the high level thinking.
Re: After two years of vibecoding, I'm back to writing by hand
#28I have AI build self-contained, smallish tasks and I check everything it does to keep the result consistent with global patterns and vision.
I stay in the loop and commit often.
Looks to me like the problem a lot of people are having is that they have AI do the whole thing.
If you ask it "refactor code to be more modern", it might guess what you mean and do it in a way you like it or not, but most likely it won't.
If you keep tasks small and clearly specced out it works just fine. A lot better than doing it by hand in many cases, specially for prototyping.
Re: After two years of vibecoding, I'm back to writing by hand
#29AI is incredibly dangerous because it can do the simple things very well, which prevents new programmers from learning the simple things ("Oh, I'll just have AI generate it") which then prevents them from learning the middlin' and harder and meta things at a visceral level. I'm a CS teacher, so this is where I see a huge danger right now and I'm explicit with my students about it: you HAVE to write the code. You CAN'…
I'm an external examiner for CS students in Denmark and I disagree with you. What we need in the industry is software engineers who can think for themselves, can interact with the business and understand it's needs, and, they need to know how computers work. What we get are mass produced coders who have been taught some outdated way of designing and building software that we need to hammer out of them. I don't partic…
Re: After two years of vibecoding, I'm back to writing by hand
#30I'm impressed that this person has been vibecoding longer than vibecoding has been a thing. A real trailblazer!
GitHub copilot was released in 2021, and Cursor was released around October 2023[0]. [0]: https://news.ycombinator.com/item?id=37888477