Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A look at Cloudflare's AI-coded OAuth library

neilmadden.blog

Post reply on HN
Print view View original

101–110 of 170 posts

Re: A look at Cloudflare's AI-coded OAuth library

#101

Post by belter »

diggan wrote: post #77
belter wrote: post #72

Earlier quoted context omitted.

You are ignoring the fact that the types of mistakes or lies are of a different nature. If you are in class, and you incorrectly argue, there is a mistake in an explanation of Derivatives or Physics, but you are the one in error, your Teacher hopefully, will not say: "Oh, I am sorry you are absolutely correct. Thank you for your advice.."

Yeah, no of course if I'm wrong I don't expect the teacher to agree with me, what kind of argument is that? I thought it was clear, but the base premise of my previous comment is that the teacher is incorrect and refuse corrections...

My point is a teacher will not do something like this:

- Confident synthesis of incompatible sources: LLM: “Einstein won the 1921 Nobel Prize for his theory of relativity, which he presented at the 1915 Solvay Conference.”

Or

- Fabricated but plausible citations: LLM: “According to Smith et al., 2022, Nature Neuroscience, dolphins recognise themselves in mirrors.” There is no such paper...model invents both authors and journal reference

And this is the danger of coding with LLMs....

Re: A look at Cloudflare's AI-coded OAuth library

#102

Post by sroussey »

ativzzz wrote: post #92
evnu wrote: post #69

Earlier quoted context omitted.

> Instead you should use high quality sources, then ask the LLM to summarize them for you to start with (NotebookLM does this very well for instance, but so can others). How do you determine if the LLM accurately reflects what the high-quality source contains, if you haven't read the source? When learning from humans, we put trust on them to teach us based on a web-of-trust. How do you determine the level of trust wi…

Because summarizing is one of the few things LLMs are generally pretty good at. Plus you should use the summary to determine if you want to read the full source, kind of like reading an abstract for a research paper before deciding if you want to read the whole thing. Bonus: the high quality source is going to be mostly AI written anyway

Actually, LLMs aren’t that great for summarizing. It would be a boon for RAG workflows if they were.

I’m still on the lookout for a great model for this.

Re: A look at Cloudflare's AI-coded OAuth library

#103

Post by ghuntley »

afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

See also: LLMs are mirrors of operator skill - https://ghuntley.com/mirrors

Re: A look at Cloudflare's AI-coded OAuth library

#104

Post by diggan »

belter wrote: post #101
diggan wrote: post #77

Earlier quoted context omitted.

Yeah, no of course if I'm wrong I don't expect the teacher to agree with me, what kind of argument is that? I thought it was clear, but the base premise of my previous comment is that the teacher is incorrect and refuse corrections...

My point is a teacher will not do something like this: - Confident synthesis of incompatible sources: LLM: “Einstein won the 1921 Nobel Prize for his theory of relativity, which he presented at the 1915 Solvay Conference.” Or - Fabricated but plausible citations: LLM: “According to Smith et al., 2022, Nature Neuroscience, dolphins recognise themselves in mirrors.” There is no such paper...model invents both authors a…

I don't know what reality you live in, but it happens that teachers are incorrect, no matter what your own personal experience have been. I'm not sure how this is even up for debate.

What matters is how X reacts when you point out it wasn't correct, at least in my opinion, and was the difference I was trying to highlight.

Re: A look at Cloudflare's AI-coded OAuth library

#105

Post by perching_aix »

evnu wrote: post #69
sulam wrote: post #62

Earlier quoted context omitted.

If you trust everything the LLM tells you, and you learn from code, then yes the same exact risks apply. But this is not how you use (or should use) LLMs when you’re learning a topic. Instead you should use high quality sources, then ask the LLM to summarize them for you to start with (NotebookLM does this very well for instance, but so can others). Then you ask it to build you a study plan, with quizzes and exercise…

> Instead you should use high quality sources, then ask the LLM to summarize them for you to start with (NotebookLM does this very well for instance, but so can others). How do you determine if the LLM accurately reflects what the high-quality source contains, if you haven't read the source? When learning from humans, we put trust on them to teach us based on a web-of-trust. How do you determine the level of trust wi…

> When learning from humans, we put trust on them to teach us based on a web-of-trust.

But this is only part of the story. When learning from another human, you'll also actively try and devise whether they're trustworthy based on general linguistic markers, and will try to find and poke holes in what they're saying so that you can question intelligently.

This is not much different from what you'd do with an LLM, which is why it's such a problem that they're more convincing than correct pretty often. But it's not an insurmountable issue. The other issue is that their trustworthiness will wary in a different way than a human's, so you need experience to know when they're possibly just making things up. But just based on feel, I think this experience is definitely possible to gain.

Re: A look at Cloudflare's AI-coded OAuth library

#107

Post by perching_aix »

blibble wrote: post #54
svara wrote: post #32

Earlier quoted context omitted.

LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…

how do you gain anything useful from a sycophantic tutor that agrees with everything you say, having being trained to behave as if the sun shines out of your rear end? making mistakes is how we learn, and if they are never pointed out...

> from a sycophantic tutor that agrees with everything you say

You know that it's possible to ask models for dissenting opinions, right? Nothing's stopping you.

> and if they are never pointed out...

They do point out mistakes though?

Re: A look at Cloudflare's AI-coded OAuth library

#108

Post by therealpygon »

svara wrote: post #32
donatj wrote: post #24

Earlier quoted context omitted.

My question is kind of in this brave new world, where do the domain experts come from? Whose going to know this stuff?

LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…

And yet, human coders may do that exact type of thing daily, producing far worse code. I find it humorous at how much higher of a standard is applied to LLMs in every discussion when I can guarantee those exact some coders likely produce their own bug-riddled software.

We’ve gone from skeptics saying LLMs can’t code, to they can’t code well, to they can’t produce human-level code, to they are riddled with hallucinations, to now “but they can’t one-shot code a library without any bugs or flaws” and “but they can only one-shot code, they can’t edit well” even tho recents coding utilities have been proving that wrong as well. And still they say they are useless.

Some people just don’t hear themselves or see how AI is constantly moving their bar.

Re: A look at Cloudflare's AI-coded OAuth library

#109

Post by loandbehold »

afro88 wrote: post #18

> What this interaction shows is how much knowledge you need to bring when you interact with an LLM. The “one big flaw” Claude produced in the middle would probably not have been spotted by someone less experienced with crypto code than this engineer obviously is. And likewise, many people would probably not have questioned the weird choice to move to PBKDF2 as a response For me this is the key takeaway. You gain pro…

Over time AI coding tools will be able to research domain knowledge. Current "AI Research" tools are already very good at it but they are not integrated with coding tools yet. The research could look at both public Internet as well as company documents that contain internal domain knowledge. Some of the domain knowledge is only in people's heads. That would need to be provided by the user.

Re: A look at Cloudflare's AI-coded OAuth library

#110

Post by brookst »

therealpygon wrote: post #108
svara wrote: post #32

Earlier quoted context omitted.

LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…

And yet, human coders may do that exact type of thing daily, producing far worse code. I find it humorous at how much higher of a standard is applied to LLMs in every discussion when I can guarantee those exact some coders likely produce their own bug-riddled software. We’ve gone from skeptics saying LLMs can’t code, to they can’t code well, to they can’t produce human-level code, to they are riddled with hallucinati…

And now the complaint is that the bugs are too subtle. Soon it will be that the overall quality is too high, leading to a false sense of security.
Post reply on HN