Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

150k lines of vibe coded Elixir: The good, the bad and the ugly

getboothiq.com

Post reply on HN
Print view View original

71–80 of 91 posts

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#71

Post by pjmlp »

michaelcampbell wrote: post #58
pjmlp wrote: post #46

Earlier quoted context omitted.

Remember, there used to be a time programmers productivity was measured in LoC per hour. As such, this is high productivity! /s

> Remember, there used to be a time programmers productivity was measured in LoC per hour. Do you remember such a time or company? I have been developing professionally since the early 1990's (and hobbyist before then), and this "truth" has been a meme even back then. I'm sure it happened, but I'm not sure it was ever as widespread as this legend would make it sound. But, there were decades of programmers programming…

I do, besides the sibling comment, there is hacker lore about these kind of issues,

> They devised a form that each engineer was required to submit every Friday, which included a field for the number of lines of code that were written that week.

https://www.folklore.org/Negative_2000_Lines_Of_Code.html

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#72

Post by nprateem »

ogig wrote: post #66
tossandthrow wrote: post #52

Earlier quoted context omitted.

What is your secret sauce? How do you organize your project?

I decided to really learn what is going on, started with: https://karpathy.ai/zero-to-hero.html That give a useful background into understanding what the tool can do, what context is, and how models are post trained. Context management is an important concept. Then I gave a shot to several tools, including copilot and gemini, but followed the general advice to use Claude Code. It's way better that the rest at the mom…

It is until it's not. That's the problem. The AI gets tripped up at some point, starts frigging tests instead of actually fixing bugs, starts looping then after several hours says it's not possible. If you're lucky.

Then on average your velocity is little better than if you just did it all by hand.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#73

Post by ogig »

nprateem wrote: post #72
ogig wrote: post #66

Earlier quoted context omitted.

I decided to really learn what is going on, started with: https://karpathy.ai/zero-to-hero.html That give a useful background into understanding what the tool can do, what context is, and how models are post trained. Context management is an important concept. Then I gave a shot to several tools, including copilot and gemini, but followed the general advice to use Claude Code. It's way better that the rest at the mom…

It is until it's not. That's the problem. The AI gets tripped up at some point, starts frigging tests instead of actually fixing bugs, starts looping then after several hours says it's not possible. If you're lucky. Then on average your velocity is little better than if you just did it all by hand.

The AI gets tripped phenomenon is something I've experienced, and I think it's again related to context usage. Using more agents and skills will reduce the pollution on the main context, and delay the moment where things go weird. /clear after each small mission. As said above, CC needs heavy guidance, but even with these issues, I'm way faster.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#74

Post by dkersten »

viktorcode wrote: post #19

It's the second time today when I see that the higher number of LoC is served as something positive. I would put it strictly in "Ugly" category. I understand the business logic that says that as long as you can vibe code away from any problems, what's the point of even looking at the code.

As the saying goes:

   Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs.
150k sounds like a lot. I do have to wonder what the program does exactly to see if that’s warranted, but it sounds bloated.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#75

Post by OoooooooO »

williamcotton wrote: post #42
miningape wrote: post #38

Earlier quoted context omitted.

Yes, as we all know, when evaluating which programming language to use, you should get a line count of the compiler's repo. More lines = more capabilities. Why would I ever want a language with less capabilities?

I mean, awk? jq? SQL?

APL

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#76

Post by logicprog »

dnautics wrote: post #63
logicprog wrote: post #2

It's interesting that Claude is able to effectively write Elixir, even if it isn't super idiomatic without established styles in the codebase, considering Elixir is a pretty niche and relatively recent language. What I'd really like to see though is experiments on whether you can few shot prompt an AI to in-context-learn a new language with any level of success.

I gave a talk about this. Without evidence, I suspect it's due to the "poisoning" phenomenon, only a few examples (~250 IIRC) is enough to push the needle, seemingly independent of LLM parameter count. Elixir has some really high quality examples available so, there is likely a "positive poisoning" effect.

I'd love to see that talk!

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#77

Post by MikeNotThePope »

vladraz wrote: post #51
vmg12 wrote: post #25

Earlier quoted context omitted.

Why not just write to the db? Just make every test independent, use uuids / random ids for ids.

Frankly this is the better solution for async tests. If the app can handle multiple users interacting with it simultaneously, then it can handle multiple tests. If it can’t, then the dev has bigger problems. As for assertions, it’s not that hard to think of a better way to check if you made an insertion or not into the db without writing “assert user_count() == 0”

I don’t disagree with you, but there are diminishing returns on making your test suite complex. To make async test work properly, you need to know what you’re doing in regards to message passing, OTP, mocks, shared memory, blah blah blah. It can get really complicated, and it is still isn’t a substitute for real user traffic. You’re going to have to rely on hiring experienced Elixir developers (small talent pool), allow for long onboarding time (expensive), or provide extensive training (difficult). Personally for most cases, writing a sync test suite and just optimizing to keep it not to slow is probably more practical in the long term.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#78

Post by dnautics »

pmontra wrote: post #69
dnautics wrote: post #61

Earlier quoted context omitted.

yes, Now this test also has to check that your redis-based cache is populated correctly. And/or sends stuff down your RabbitMQ/Kafka pipeline.

That looks like an integration test. A possible way to handle that scenario is to drop all the databases after it ends and create them again, or truncate all the tables or whatever it makes sense for that possible set of different data stores. That could run on developer machines but maybe it runs only on a CI server and developers run only unit tests.

so in elixir you can do this async alongside your unit tests.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#79

Post by Winsaucerer »

pmontra wrote: post #11

> In Elixir tests, each test runs in a database transaction that rolls back at the end. Tests run async without hitting each other. No test data persists. And it confuses Claude. This way of running tests is also what Rails does, and AFAIK Django too. Tests are isolated and can be run in random order. Actually, Rails randomizes the order so if the are tests that for any reason depend on the order of execution, they w…

Some database tests can't be done within transactions. With postgres, I create a copy of the database via WITH TEMPLATE, and each test runs in its own copy of the database. Then it can use or avoid transactions as it pleases, because the whole thing is local to that one tests anyway.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#80

Post by whalesalad »

pmontra wrote: post #11

> In Elixir tests, each test runs in a database transaction that rolls back at the end. Tests run async without hitting each other. No test data persists. And it confuses Claude. This way of running tests is also what Rails does, and AFAIK Django too. Tests are isolated and can be run in random order. Actually, Rails randomizes the order so if the are tests that for any reason depend on the order of execution, they w…

This is a skill issue in prompt engineering. You explain how that works in CLAUDE.md and it is a non issue.
Post reply on HN