A look at Cloudflare's AI-coded OAuth library
neilmadden.blog
A look at Cloudflare's AI-coded OAuth library
1–10 of 170 posts
Re: A look at Cloudflare's AI-coded OAuth library
#2Mostly a good writeup, but I think there's some serious shifting the goalposts of what "vibe coded" means in a disingenuous way towards the end:
'Yes, this does come across as a bit “vibe-coded”, despite what the README says, but so does a lot of code I see written by humans. LLM or not, we have to give a shit.'
If what most people do is "vibe coding" in general, the current definition of vibe coding is essentially meaningless. Instead, the author is making the distinction between "interim workable" and "stainless/battle tested" which is another dimension of code entirely. To describe that as vibe coding causes me to view the author's intent with suspicion.
Re: A look at Cloudflare's AI-coded OAuth library
#3Mostly a good writeup, but I think there's some serious shifting the goalposts of what "vibe coded" means in a disingenuous way towards the end: 'Yes, this does come across as a bit “vibe-coded”, despite what the README says, but so does a lot of code I see written by humans. LLM or not, we have to give a shit.' If what most people do is "vibe coding" in general, the current definition of vibe coding is essentially m…
Isn’t vibe coding just C&P from AI instead of Stack Overflow?
I read it as: done by AI but not checked by humans.
Re: A look at Cloudflare's AI-coded OAuth library
#4Mostly a good writeup, but I think there's some serious shifting the goalposts of what "vibe coded" means in a disingenuous way towards the end: 'Yes, this does come across as a bit “vibe-coded”, despite what the README says, but so does a lot of code I see written by humans. LLM or not, we have to give a shit.' If what most people do is "vibe coding" in general, the current definition of vibe coding is essentially m…
I find ”vibe coding” to be one of the, if not the, concepts in this business to lose its meaning the fastest. Similar to how everything all of a sudden was ”cloud” now everything is ”vibe coded”, even though reading the original tweet really narrows it down thoroughly.
Re: A look at Cloudflare's AI-coded OAuth library
#5A very good piece that clearly illustrates one of the dangers with LLS's: responsibility for code quality is blindly offloaded on the automatic system
> There are some tests, and they are OK, but they are woefully inadequate for what I would expect of a critical auth service. Testing every MUST and MUST NOT in the spec is a bare minimum, not to mention as many abuse cases as you can think of, but none of that is here from what I can see: just basic functionality tests.
and
> There are some odd choices in the code, and things that lead me to believe that the people involved are not actually familiar with the OAuth specs at all. For example, this commit adds support for public clients, but does so by implementing the deprecated “implicit” grant (removed in OAuth 2.1).
As Madden concludes "LLM or not, we have to give a shit."
Re: A look at Cloudflare's AI-coded OAuth library
#6> At ForgeRock, we had hundreds of security bugs in our OAuth implementation, and that was despite having 100s of thousands of automated tests run on every commit, threat modelling, top-flight SAST/DAST, and extremely careful security review by experts.
Wow. Anecdotally it's my understanding that OAuth is ... tricky ... but wow.
Some would say it's a dumpster fire. I've never read the spec or implemented it.
Re: A look at Cloudflare's AI-coded OAuth library
#7Really interesting breakdown. What jumped out to me wasn’t just the bugs (CORS wide open, incorrect Basic auth, weak token randomness), but how much the human devs seemed to lean on Claude’s output even when it was clearly offbase. That “implicit grant for public clients” bit is wild; it’s deprecated in OAuth 2.1, and Claude just tossed it in like it was fine, and then it stuck.
Re: A look at Cloudflare's AI-coded OAuth library
#8I suggest they freeze a branch of it, then spawn some AIs to introduce and attempt to hide vulnerabilities, and another to spot and fix them. Every commit is a move, and try to model the human evolution of chess.
Re: A look at Cloudflare's AI-coded OAuth library
#9> Another hint that this is not written by people familiar with OAuth is that they have implemented Basic auth support incorrectly.
so tldr most of the issue the author has is against the person who made the library is the design not the implementation?
Re: A look at Cloudflare's AI-coded OAuth library
#10"...A more serious bug is that the code that generates token IDs is not sound: it generates biased output. This is a classic bug when people naively try to generate random strings, and the LLM spat it out in the very first commit as far as I can see. I don’t think it’s exploitable: it reduces the entropy of the tokens, but not far enough to be brute-forceable. But it somewhat gives the lie to the idea that experienced security professionals reviewed every line of AI-generated code...."
In the Github repo Cloudflare says:
"...Claude's output was thoroughly reviewed by Cloudflare engineers with careful attention paid to security and compliance with standards..."
My conclusion is that as a development team, they learned little since 2017: https://news.ycombinator.com/item?id=13718752