Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

150k lines of vibe coded Elixir: The good, the bad and the ugly

getboothiq.com

Post reply on HN
Print view View original

61–70 of 91 posts

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#61

Post by dnautics »

christophilus wrote: post #59
mystifyingpoi wrote: post #40

Earlier quoted context omitted.

> Just make every test independent That's easier said than done. Simple example: API that returns a count of all users in the database. The obvious correct implementation that will work would be just to `select count(*) from users`. But if some other test touches users table beforehand, it won't work. There is no uuid to latch onto here.

That’s why you run each test in a transaction with proper isolation level, and don’t commit the transaction— roll it back when the test ends. No test ever interferes with another that way.

yes, Now this test also has to check that your redis-based cache is populated correctly. And/or sends stuff down your RabbitMQ/Kafka pipeline.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#62

Post by dnautics »

tossandthrow wrote: post #9

It seems like the 100% vibe coded is an exaggeration given that Claude fails at certain tasks. The new generation of code assistants are great. But when I dogmatically try to only let the AI work on a project it usually fails and shots itself in its proverbial feet. If this is indeed 100% vibe coded, then there is some magic I would love to learn!

I think by 100% vibe coded most people on hn mean that 100% of the code is written not by hand. The hand only does the delete key and prompting. We're mostly not talking about amateurs with no CS background just prompting and shitting out software with all sorts of bugs they would never be able to see.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#63

Post by dnautics »

logicprog wrote: post #2

It's interesting that Claude is able to effectively write Elixir, even if it isn't super idiomatic without established styles in the codebase, considering Elixir is a pretty niche and relatively recent language. What I'd really like to see though is experiments on whether you can few shot prompt an AI to in-context-learn a new language with any level of success.

I gave a talk about this. Without evidence, I suspect it's due to the "poisoning" phenomenon, only a few examples (~250 IIRC) is enough to push the needle, seemingly independent of LLM parameter count. Elixir has some really high quality examples available so, there is likely a "positive poisoning" effect.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#64

Post by dnautics »

pmontra wrote: post #11

> In Elixir tests, each test runs in a database transaction that rolls back at the end. Tests run async without hitting each other. No test data persists. And it confuses Claude. This way of running tests is also what Rails does, and AFAIK Django too. Tests are isolated and can be run in random order. Actually, Rails randomizes the order so if the are tests that for any reason depend on the order of execution, they w…

> And it confuses Claude.

I've never had this problem.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#65

Post by davidclark »

Sharlin wrote: post #56

I don’t understand how the author can simultaneously argue that Claude is great at Elixir because it’s a small language with only one paradigm, and also that Claude is bad at Elixir, spewing out non-idiomatic code that makes little sense in the functional paradigm?

The secret is that the author is also Claude.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#66

Post by ogig »

tossandthrow wrote: post #52
ogig wrote: post #15

Earlier quoted context omitted.

My last two projects have been 100% coded using Claude, and one has certain complexity. I don't think there is coming back for me.

What is your secret sauce? How do you organize your project?

I decided to really learn what is going on, started with: https://karpathy.ai/zero-to-hero.html That give a useful background into understanding what the tool can do, what context is, and how models are post trained. Context management is an important concept. Then I gave a shot to several tools, including copilot and gemini, but followed the general advice to use Claude Code. It's way better that the rest at the moment. And then I dive deep into Claude Code documentation and different youtube videos, there is plenty of good content out there. There are some ways to customize and increase the determinism of the process by using the tools properly.

Overall my process is, define a broad spec, including architecture. Heavy usage of standard libraries and frameworks is very helpful, also typed languages. Create skills according to your needs, and use MCP to give CC a feedback mechanism, playwright is a must for web development.

After the environment and initial seed is in place in the form of a clear spec, it's process of iteration via conversation. My session tend to go "Lets implement X, plan it", CC offers a few route, I pick what makes most sense, or on occasions I need to explain the route I want to take. After the feature is implemented we go into a cleanup phase, we check if anything might be going out of hand, recheck security stuff, and create testing. Repeat. Pick small battles, instead of huge features. I'm doing quite a lot of hand handling at the moment, saying a lots of "no", but the process is on another level with what I was doing before, and the speed I can get features out is insane.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#67

Post by troupo »

d3ckard wrote: post #10
logicprog wrote: post #2

It's interesting that Claude is able to effectively write Elixir, even if it isn't super idiomatic without established styles in the codebase, considering Elixir is a pretty niche and relatively recent language. What I'd really like to see though is experiments on whether you can few shot prompt an AI to in-context-learn a new language with any level of success.

I would argue effectiveness point. It's certainly helpful, but has a tendency to go for very non idiomatic patterns (like using exceptions for control flow). Plus, it has issues which I assume are the effect of reinforcement learning - it struggles with letting things crash and tends to silence things that should never fail silently.

> has a tendency to go for very non idiomatic patterns (like using exceptions for control flow).

It tends to always write Java even if it's Elixir. Usage rules help: https://hexdocs.pm/usage_rules/readme.html

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#68

Post by kryptiskt »

michaelcampbell wrote: post #58
pjmlp wrote: post #46

Earlier quoted context omitted.

Remember, there used to be a time programmers productivity was measured in LoC per hour. As such, this is high productivity! /s

> Remember, there used to be a time programmers productivity was measured in LoC per hour. Do you remember such a time or company? I have been developing professionally since the early 1990's (and hobbyist before then), and this "truth" has been a meme even back then. I'm sure it happened, but I'm not sure it was ever as widespread as this legend would make it sound. But, there were decades of programmers programming…

IBM had such a culture back in the day, where they feted 1 kloc/day programmers. That was what Bill Gates sneered at with the "Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs" quote.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#69

Post by pmontra »

dnautics wrote: post #61
christophilus wrote: post #59

Earlier quoted context omitted.

That’s why you run each test in a transaction with proper isolation level, and don’t commit the transaction— roll it back when the test ends. No test ever interferes with another that way.

yes, Now this test also has to check that your redis-based cache is populated correctly. And/or sends stuff down your RabbitMQ/Kafka pipeline.

That looks like an integration test. A possible way to handle that scenario is to drop all the databases after it ends and create them again, or truncate all the tables or whatever it makes sense for that possible set of different data stores.

That could run on developer machines but maybe it runs only on a CI server and developers run only unit tests.

Re: 150k lines of vibe coded Elixir: The good, the bad and the ugly

#70

Post by idle_zealot »

esafak wrote: post #34
viktorcode wrote: post #19

It's the second time today when I see that the higher number of LoC is served as something positive. I would put it strictly in "Ugly" category. I understand the business logic that says that as long as you can vibe code away from any problems, what's the point of even looking at the code.

Think of it as 60 man-years of work.

If that's true then I can ship 60 man-years of work with

  yes 'println("a very important and useful line of code");' >> main.c
in under a second!
Post reply on HN