Isn't that like proofreading text in a language you're not familiar with?
Ask HN: How do you audit LLM code in programming languages you don't know?
11–19 of 19 posts
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#12So before the gate keeping starts, my first crack at optimizations was in 1987 on a 1 MHz Apple //e
1. I wrote the code in BASIC
2. I wrote the code in assembly
3. I got more improvement because storing and reading from the first page of memory took two clock cycles instead of 3.
But this isn’t 1986, this is 2026. I “vibe coded” my first project this year. I designed the AWS architecture from the an empty account using IAC, I chose every service to use, I verified every permission, I chose and designed the orchestration, the concurrency model, I gathered requirements. What I didn’t do is look at a line of Python code or infrastructure code aside from the permissions that Codex generated.
Now to answer your questions :
How did I validate the correctness? Just like if I had written it myself. I had Codex to create a shell script to do end to end tests of all of the scenarios I cared about and when one broke, I went back to Codex to fix it. I was very detailed about the scenarios.
The web front end that I used was built by another developer. I haven’t touched web dev in a decade. I told Codex what changes I needed and I verified the changes by deploying it and testing it manually.
How did I validate the performance? Again just like I would do on something I wrote. I tested it first with a few hundred transactions to verify the functionality and then I stress tested it with a real world amount a transactions. The first iteration broke horribly. Not because of Claude code. It was a bad design.
But here’s the beauty. It took me a day to do the bad implementation that would have taken me three or four days. I then redesigned it, didn’t use the AWS service and did I designed that was much more scalable and it took a day. I knew in theory how it worked under the hood. But not in practice. Again I tested for scalability by testing the result.
The architectural quality? I validated it by synthesizing real world traffic. ChatGPT in thinking mode did find a subtle concurrency bug. That was my fault though. I designed the concurrency implementation, Codex just did what I told it to do.
Subtle bugs happen whether people write it or an agent writes it. You do the best you can with your tests and when they come up you fix it?
How do I prevent technical debt? All large implementations have technical debt. Again just like when I lead a team - I componenitize everything with clean interfaces. It makes it easier for coding agents and people.
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#13[flagged]
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#14I prefer the ancient Chinese science of Oracle Bone divination. You take the scapulae of an ox and copy the PR diff onto the bone using jiǎgǔwén encoding, then throw it in a fire until thermal expansion causes the bone to crack. You then take a photo of the cracked bone and feed it back to your coding agent, which has been properly trained in interpreting Oracle Bones to extract PR review comments. If the PR is too b…
Jiǎgǔwén has been used for decision making for 3000 years. It's a stable technology that just works.
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#15One thing I’ve found helpful is relying less on understanding the syntax itself and more on validating behavior at the system level. Even if I don’t know the language well, I can still focus on things like test coverage, edge cases, performance patterns, and whether the architecture follows familiar design principles. In that sense it ends up feeling more like auditing a black box than reviewing code line by line.
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#16[flagged]
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#17Treating generated code in unknown languages as a black box creates significant risk, particularly regarding security vulnerabilities or race conditions that functional tests often miss. If that unvetted code causes data corruption or a production outage, how do you handle the immediate remediation and liability without the internal expertise to debug it? Have you considered using a secondary, distinct model specifically prompted to act as an adversarial "inspector" to critique the architectural decisions of the first? I'm curious if you rely solely on end-to-end testing or if you implement strict sandboxing to limit the blast radius of code you can't manually review.
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#18Isn't that like proofreading text in a language you're not familiar with?
I was thinking the same thing. The number of vibe-coded language learning apps I've seen where the developer wasn't fluent in the target languages just makes me shake my head in disbelief.
Re: Ask HN: How do you audit LLM code in programming languages you don't know?
#19I built a tool that detects dead code across 26 languages.
The honest answer is most people aren't catching it. The code works, the tests pass, and nobody reviews what the LLM left behind. You end up shipping unreachable functions, duplicate logic, and unused imports that sit there unreviewed and unpatched.
Dead code isn't just technical debt - MITRE catalogued it as CWE-561, a security weakness. A duplicated goto fail; once bypassed SSL on 500M+ Apple devices. It's a hidden attack surface that never gets looked at.
If you're vibe-coding regularly, I'd treat every LLM output like a junior dev's PR. Assume it works, assume it's not optimal, and run static analysis as a baseline. For dead code specifically check out my tool at swynx.io - plenty of free options.