Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A look at Cloudflare's AI-coded OAuth library

neilmadden.blog

Post reply on HN
Print view View original

31–40 of 170 posts

Re: A look at Cloudflare's AI-coded OAuth library

#31

Post by jajko »

djoldman wrote: post #6

> At ForgeRock, we had hundreds of security bugs in our OAuth implementation, and that was despite having 100s of thousands of automated tests run on every commit, threat modelling, top-flight SAST/DAST, and extremely careful security review by experts. Wow. Anecdotally it's my understanding that OAuth is ... tricky ... but wow. Some would say it's a dumpster fire. I've never read the spec or implemented it.

Hundreds of thousands of tests? That sounds like quantity > quality or outright llm-generated ones, who even maintains them?

Re: A look at Cloudflare's AI-coded OAuth library

#32

Post by svara »

donatj wrote: post #24
afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?

LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains.

The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly.

The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don't do that.

But if you're willing to learn, this is rocket fuel.

Re: A look at Cloudflare's AI-coded OAuth library

#33

Post by ants_everywhere »

ajmurmann wrote: post #29
afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

I've been using an llm to do much of a k8s deployment for me. It's quick to get something working but I've had to constantly remind it to use secrets instead of committing credentials in clear text. A dangerous way to fail. I wonder if in my case this is caused by the training data having lots of examples from online tutorials that omit security concerns to focus on the basics.

> It's quick to get something working but I've had to constantly remind it to use secrets instead of committing credentials in clear text.

This is going to be a powerful feedback loop which you might call regression to the intellectual mean.

On any task, most training data is going to represent the middle (or beginning) of knowledge about a topic. Most k8s examples will skip best practices, most react apps will be from people just learning react, etc.

If you want the LLM to do best practices in every knowledge domain (assuming best practices can be consistently well defined), then you have to push it away from the mean of every knowledge domain simultaneously (or else work with specialized fine tuned models).

As you continue to add training data it will tend to regress toward the middle because that's where most people are on most topics.

Re: A look at Cloudflare's AI-coded OAuth library

#35

Post by bradly »

afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

I've found llms are very quick to add defaults, fallbacks, rescues–which all makes it very easy for code to look like it is working when it is not or will not. I call this out three different places in my CLAUDE.md trying to adjust for this, and still occasionally get.

Re: A look at Cloudflare's AI-coded OAuth library

#36

Post by qzw »

shswkna wrote: post #28
donatj wrote: post #24

Earlier quoted context omitted.

My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?

Most important question on this entire topic. Fast forward 30 years and modern civilisation is entirely dependent on our AI’s. Will deep insight and innovation from a human perspective perhaps come to a stop?

No, but it'll become a hobby or artistic pursuit, just like running, playing chess, or blacksmithing. But I personally think it's going to take longer than 30 years.

Re: A look at Cloudflare's AI-coded OAuth library

#37

Post by _heimdall »

marcusb wrote: post #22
afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

I’m puzzled when I hear people say ‘oh, I only use LLMs for things I don’t understand well. If I’m an expert, I’d rather do it myself.’ In addition to the ability to review output effectively, I find the more closely I’m able to describe what I want in the way another expert in that domain would, the better the LLM output. Which isn’t really that surprising for a statistical text generation engine.

That's why outsource most other things in our life though, why would it be different with LLMs?

People don't learn how a car works before buying one, they just take it to a mechanic when it breaks. Most people don't know how to build a house, they have someone else build it and assume it was done well.

I fully expect people to similarly have LLMs do what the person doesn't know how and assume the machine knew what to do.

Re: A look at Cloudflare's AI-coded OAuth library

#38

Post by diggan »

marcusb wrote: post #22
afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

I’m puzzled when I hear people say ‘oh, I only use LLMs for things I don’t understand well. If I’m an expert, I’d rather do it myself.’ In addition to the ability to review output effectively, I find the more closely I’m able to describe what I want in the way another expert in that domain would, the better the LLM output. Which isn’t really that surprising for a statistical text generation engine.

I guess it depends. In some cases, you don't have to understand the black box code it gives you, just that it works within your requirements.

For example, I'm horrible at math, always been, so writing math-heavy code is difficult for me, I'll confess to not understanding math well enough. If I'm coding with an LLM and making it write math-heavy code, I write a bunch of unit tests to describe what I expect the function to return, write a short description and give it to the LLM. Once the function is written, run the tests and if it passes, great.

I might not 100% understand what the function does internally, and it's not used for any life-preserving stuff either (typically end up having to deal with math for games), but I do understand what it outputs, and what I need to input, and in many cases that's good enough. Working in a company/with people smarter than you tends to make you end up in this situation anyways, LLMs or not.

Though if in the future I end up needing to change the math-heavy stuff in the function, I'm kind of locked into using LLMs for understanding and changing it, which obviously feels less good. But the alternative is not doing it at all, so another tradeoff I suppose.

I still wouldn't use this approach for essential/"important" stuff, but more like utility functions.

Re: A look at Cloudflare's AI-coded OAuth library

#39

Post by keybored »

techpression wrote: post #4
CuriouslyC wrote: post #2

Mostly a good writeup, but I think there's some serious shifting the goalposts of what "vibe coded" means in a disingenuous way towards the end: 'Yes, this does come across as a bit “vibe-coded”, despite what the README says, but so does a lot of code I see written by humans. LLM or not, we have to give a shit.' If what most people do is "vibe coding" in general, the current definition of vibe coding is essentially m…

I find ”vibe coding” to be one of the, if not the, concepts in this business to lose its meaning the fastest. Similar to how everything all of a sudden was ”cloud” now everything is ”vibe coded”, even though reading the original tweet really narrows it down thoroughly.

Viral marketing campaign term losing its meaning makes sense.

Re: A look at Cloudflare's AI-coded OAuth library

#40

Post by diggan »

ajmurmann wrote: post #29
afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

I've been using an llm to do much of a k8s deployment for me. It's quick to get something working but I've had to constantly remind it to use secrets instead of committing credentials in clear text. A dangerous way to fail. I wonder if in my case this is caused by the training data having lots of examples from online tutorials that omit security concerns to focus on the basics.

> my case this is caused by the training data having

I think it's caused by you not having a strong enough system prompt. Once you've built up a slightly reusable system prompt for coding or for infra work, where you bit by bit build it up while using a specific model (since different models respond differently to prompts), you end up getting better and better responses.

So if you notice it putting plaintext credentials in the code, add to the system prompt to not do that. With LLMs you really get what you ask for, and if you miss to specify anything, the LLM will do whatever the probabilities tells it to, but you can steer this by being more specific.

Imagine you're talking to a very literal and pedantic engineer who argues a lot on HN and having to be very precise with your words, and you're like 80% of the way there :)

Post reply on HN