Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A look at Cloudflare's AI-coded OAuth library

neilmadden.blog

Post reply on HN
Print view View original

111–120 of 170 posts

Re: A look at Cloudflare's AI-coded OAuth library

#111

Post by diogocp »

threeseed wrote: post #82
svara wrote: post #32

Earlier quoted context omitted.

LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…

Learning from LLMs is akin to learning from Joe Rogan. You are getting a stylised view of a topic from an entity who lacks the deep understanding needed to be able to fully distill the information. But it is enough to gain enough knowledge for you to feel confident which is still valuable but also dangerous. And I assure you that many, many people are delegating to LLMs blindly e.g. it's a huge problem in the UK lega…

> You are getting a stylised view of a topic from an entity who lacks the deep understanding

Isn't this how every child learns?

Unless his father happens to be king of Macedonia, of course.

Re: A look at Cloudflare's AI-coded OAuth library

#113

Post by brookst »

skeeter2020 wrote: post #83
signatoremo wrote: post #73

Earlier quoted context omitted.

And you are saying human teachers or online materials won't lie to you once or twice for every 20 facts? no matter how small. Did you do any comparison?

it's not jsut the lies, but how it lies and the fact that LLMs are very hesitant to call out humans on their BS

Is this the newest meme?

Me: “explain why radioactive half-life changes with temperature”

ChatGPT 4o: “ Short answer: It doesn’t—at least not significantly. Radioactive Half-Life is (Almost Always) Temperature-Independent”

…and then it goes on to give a few edge cases where there’s a tiny effect.

Re: A look at Cloudflare's AI-coded OAuth library

#114

Post by paradox242 »

donatj wrote: post #24
afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?

The implication is that they are hoping to bridge the gap between current AI capabilities and something more like AGI in the time it takes the senior engineers to leave the industry. At least, that's the best I can come up with, because they are kicking out all of the bottom rings of the ladder here in what otherwise seems like a very shortsighted move.

Re: A look at Cloudflare's AI-coded OAuth library

#115

Post by brookst »

shswkna wrote: post #28
donatj wrote: post #24

Earlier quoted context omitted.

My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?

Most important question on this entire topic. Fast forward 30 years and modern civilisation is entirely dependent on our AI’s. Will deep insight and innovation from a human perspective perhaps come to a stop?

Did musical creativity end with synths and sequencers?

Tools will only amplify human skills. Sure, not everyone will choose to use tools for anything meaningful, but those people are driving human insight and innovation today anyway.

Re: A look at Cloudflare's AI-coded OAuth library

#116

Post by kentonv »

aiono wrote: post #20

I agree with the last paragraph about doing this yourself. Humans have tendency to take shortcuts while thinking. If you see something resembling what you expect for the end product you will be much less critical of it. The looks/aesthetics matter a lot on finding problems with in a piece of code you are reading. You can verify this by injecting bugs in your code changes and see if reviewers can find them. On the oth…

I agree that most code reviewers are pretty bad at spotting subtle bugs in code that looks good superficially.

I have a lot of experience reviewing code -- more than I ever really wanted. It has... turned me cynical and bitter, to the point that I never believe anything is right, no matter who wrote it or how nice it looks, because I've seen so many ways things can go wrong. So I tend to review every line, simulate it in my head, and catch things. I kind of hate it, because it takes so long for me to be comfortable approving anything, and my reviewees hate it too, so they tend to avoid sending things to me.

I think I agree that if I'd written the code by hand, it would be less likely to have bugs. Maybe. I'm not sure, because I've been known to author some pretty dumb bugs of my own. But yes, total Kenton brain cycles spent on each line would be higher, certainly.

On the other hand, though, I probably would not have been the one to write this library. I just have too much on my plate (including all those reviews). So it probably would have been passed off to a more junior engineer, and I would have reviewed their work. Would I have been more or less critical? Hard to say.

But one thing I definitely disagree with is the idea that humans would have produced bug-free code. I've seen way too many bugs in my time to take that seriously. Hate to say it but most of the bugs I saw Claude produce are mistakes I'd totally expect an average human engineer could make.

Aside, since I know some people are thinking it: At this time, I do not believe LLM use will "replace" any human engineers at Cloudflare. Our hiring of humans is not determined by how much stuff we have to do, because we basically have infinite stuff we want to do. The limiting factor is what we have budget for. If each human becomes more productive due to LLM use, and this leads to faster revenue growth, this likely allows us to hire more people, not fewer. (Disclaimer: As with all of my comments, this is my own opinion / observation, not an official company position.)

Re: A look at Cloudflare's AI-coded OAuth library

#117

Post by conradev »

svara wrote: post #32
donatj wrote: post #24

Earlier quoted context omitted.

My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?

LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…

  Just wrong. Don’t do that
I’d personally qualify this: don’t ship that code, but absolutely do it personally to grow if you’re interested.

I’ve grown the most when I start with things I sort of know and I work to expand my understanding.

Re: A look at Cloudflare's AI-coded OAuth library

#118

Post by paradox242 »

svara wrote: post #32
donatj wrote: post #24

Earlier quoted context omitted.

My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?

LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…

The value of LLMs is that they do things for you, so yeah the incentive is to have them take over more and more of the process. I can also see a future not far into the horizon where those who grew up with nothing but AI are much less discerning and capable and so the AI becomes more and more a crutch, as human capability withers from extended disuse.

Re: A look at Cloudflare's AI-coded OAuth library

#119

Post by wslh »

loandbehold wrote: post #109
afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

Over time AI coding tools will be able to research domain knowledge. Current "AI Research" tools are already very good at it but they are not integrated with coding tools yet. The research could look at both public Internet as well as company documents that contain internal domain knowledge. Some of the domain knowledge is only in people's heads. That would need to be provided by the user.

I'd like to add a practical observation, even assuming much more capable AI in the future: not all failures are due to model limitations, sometimes it's about external [world] changes.

For instance, I used Next.js to build a simple login page with Google auth. It worked great, even though I only had basic knowledge of Node.js and a bit of React.

Then I tried adding a database layer using Prisma to persist users. That's where things broke. The integration didn't work, seemingly due to recent versions in Prisma or subtle breaking updates. I found similar issues discussed on GitHub and Reddit, but solving them required shifting into full manual debugging mode.

My takeaway: even with improved models, fast-moving frameworks and toolchains can break workflows in ways that LLMs/ML (at least today) can't reason through or fix reliably. It's not always about missing domain knowledge, it's that the moving parts aren't in sync with the model yet.

Re: A look at Cloudflare's AI-coded OAuth library

#120

Post by spenczar5 »

ape4 wrote: post #34

The article says there aren't too many useless comments but the code has: // Get the Origin header from the request const origin = request.headers.get('Origin');

Of course, these are awful for a human. But I wonder if they're actually helpful for the LLM when it's reading code. It means each line of behavior is written in two ways: human language and code. Maybe that rosetta stone helps it confidently proceed in understanding, at the cost of tokens.

All speculation, but I'd be curious to see it evaluated - does the LLM do better edits on egregiously commented code?

Post reply on HN