Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I let LLMs write an Elixir NIF in C; it mostly worked

overbring.com

Post reply on HN
Print view View original

31–40 of 73 posts

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#31

Post by brokencode »

flax wrote: post #2

"it mostly worked" is just a more nuanced way of saying "it didn't work". Apparently the author did eventually get something working, but it is false to say that the LLMs produced a working project.

Ok. But what are you even reacting to? Who is saying that it produced a working product?

As you said, the very title of the article acknowledged that it didn’t produce a working product.

This is just outrage for the sake of outrage.

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#32

Post by overbring_labs »

drumnerd wrote: post #17

I would never ever let an LLM anywhere near C code. If you need help from LLM to write a NIF that performs basic C calls to the OS, you probably can’t check if it’s safe. I mean, it needs at least to pass valgrind.

I mean, you aren't wrong. I'm looking into converting it into Rust with Rustler right now.

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#33

Post by overbring_labs »

jgalt212 wrote: post #9
flax wrote: post #2

"it mostly worked" is just a more nuanced way of saying "it didn't work". Apparently the author did eventually get something working, but it is false to say that the LLMs produced a working project.

I dunno. Depending on the writer and their particularly axe to grind the definition can vary widely. I would like it to mean, "any fixes I needed to make were minimal and not time intensive."

It's more of "yeah it worked, but I had to do a lot of hand-holding" and "it passes the tests but I cannot tell if the code has memory leaks".

Actually, I can tell; I ran split on the C source and got things like this:

disk_space.c:144:16: Only storage bin.ref_bin (type void *) derived from variable declared in this scope is not released (memory leak)

So I'm looking into a Rust version with Rustler now.

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#34

Post by overbring_labs »

flax wrote: post #2

"it mostly worked" is just a more nuanced way of saying "it didn't work". Apparently the author did eventually get something working, but it is false to say that the LLMs produced a working project.

What is your definition of "a working project"? It does what it says on the tin (actually it probably does more, because splint throws some warnings...)

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#35

Post by Alive-in-2025 »

true_religion wrote: post #22
drumnerd wrote: post #17

I would never ever let an LLM anywhere near C code. If you need help from LLM to write a NIF that performs basic C calls to the OS, you probably can’t check if it’s safe. I mean, it needs at least to pass valgrind.

Security is a spectrum. If you totally control the input going into a program, it can be safe even if you didn't test it for memory leaks. The only errors that occur will be truly erroneous, not malicious and for many solutions that's fine. At the very least, it's fine for personal projects which is something I'm getting into more and more: remembering that computers were meant to create convenience, so writing small…

For personal projects, ok security is different. But get out of that, and I'd do it even for that, you need defense in depth. You think you sanitized your input but your C program has a bug and a vulnerability - or your Java program or whatever has bugs. Almost everything has some bugs, and thus your vulnerabilities will hit eventually in your C program, even if you were careful.

I'd say absent some temporary hack to do something, my bad experiences won't let me say something is low risk. I worked at Microsoft years ago, and after the zillions of vulnerabilities were attacked by people around the time of windows 95 and computers on the net, we did serious code reviews in my team of the data access libraries. There were vast numbers of vulnerabilities. A group of 3 or 4 of us would sit in a room for 3 hours a day, one person a scribe, and we'd go over this c code that was ancient even then - we found problems everywhere, it was exhausting and shocking. The entire data access infrastructure was riddled with memory leaks, strings that were not length limited, input parameters that were not checked or sanitized, etc. I'm sure it was endemic across all components, not just there. We fixed some things, but we found so much shit.

Thank got I wasn't on the team trying to figure out what to do about those problems. I think they end of lifed a lot of stuff.

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#37

Post by simonw »

drumnerd wrote: post #17

I would never ever let an LLM anywhere near C code. If you need help from LLM to write a NIF that performs basic C calls to the OS, you probably can’t check if it’s safe. I mean, it needs at least to pass valgrind.

You can use something like Claude Code or Codex CLI and tell it to run valgrind as part of iterating on the code.

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#38

Post by napsterbr »

cess11 wrote: post #24
cpursley wrote: post #19

Earlier quoted context omitted.

Here's one Claude-vibed project that makes me money that I run in addition to my saas, which is Elixir. I'm not strong in TypeScript and this is an Astro static site, so Claude has been really helpful. Backend is Supabase (postgres) and a few background jobs via https://pgflow.dev (pgmq) that fetch and populate job openings and uses some AI steps to filter then classify into the correct categories (among other things…

Is there a reason why you're using 'when is_struct/2' instead of pattern matching here? https://github.com/cpursley/livefilter/blob/main/lib/live_fi...

This is clearly low quality, non-idiomatic AI-generated Elixir code. So the likely answer is that "you" did not use this at all; AI did.

I review this kind of AI-generated Elixir code on a daily basis. And it makes me want to go back to ~2022, when code in pull requests actually made sense.

Apologies for the rant, this is just a burnt out developer tired of reviewing this kind of code.

PS: companies should definitely highlight "No low-quality AI code" in job listings as a valid perk.

Re: I let LLMs write an Elixir NIF in C; it mostly worked

#39

Post by cpursley »

napsterbr wrote: post #38
cess11 wrote: post #24

Earlier quoted context omitted.

Is there a reason why you're using 'when is_struct/2' instead of pattern matching here? https://github.com/cpursley/livefilter/blob/main/lib/live_fi...

This is clearly low quality, non-idiomatic AI-generated Elixir code. So the likely answer is that "you" did not use this at all; AI did. I review this kind of AI-generated Elixir code on a daily basis. And it makes me want to go back to ~2022, when code in pull requests actually made sense. Apologies for the rant, this is just a burnt out developer tired of reviewing this kind of code. PS: companies should definitely…

Fwiw, the date range part of this is the lowest quality, I even have an issue open: https://github.com/cpursley/livefilter/issues/2

In production code I'd do a couple passes and tell it to lean into more function head and guard matching, etc.

But it does compiles, and works: https://livefilter.fly.dev/todos?filters%5Bassigned_to%5D%5B...

Post reply on HN