Curl: We still have not seen a valid security report done with AI help • Hacker News Board

Re: Curl: We still have not seen a valid security report done with AI help

#241

Post by alwa » Wed, May 07, 2025, 3:49 PM UTC

namaria wrote: post #228

alwa wrote: post #92

Earlier quoted context omitted.

I mean… I have a fancy phone camera in my pocket too, but there are photographers who, with the same model of fancy phone camera, do things that awe and move me. It took a solid hundred years to legitimate photography as an artistic medium, right? To the extent that the controversy still isn’t entirely dead? Any cool images I ask AI for are going to involve a lot less patience and refinement than some of these things…

> It took a solid hundred years to legitimate photography as an artistic medium, right? Not really. "In 1853 the Photographic Society, parent of the present Royal Photographic Society, was formed in London, and in the following year the Société Française de Photographie was founded in Paris." https://www.britannica.com/technology/photography/Photograph...

Not that photographic art wasn’t getting made, more that the doyens of the Finer Arts would tend to dismiss work in that medium as craft, trade, or low art—that they’d dismiss the act of photographic production as “mere capture” as opposed to creative interpretation, or situate the artistic work in the darkroom afterward where people used hands and brushes and manual aesthetic judgment.

It’s been depressingly long since school, but am I wrong in vaguely remembering the controversy stretching through Art in the Age of Mechanical Reproduction and well into the Warhol era?

https://news.harvard.edu/gazette/story/2010/10/when-photogra...

And I guess legitimacy doesn’t fully depend on the whims of museums and collectors, but to hear Christie’s tell it, they didn’t start treating the medium as fine art until 1972–and then, almost more as antiquities than as works of art—

https://www.christies.com/en/stories/how-photography-became-...

In much the same way as there are tons of Polaroids that are not art and a few that unambiguously are (e.g. [0]); there’s a lot of lazy AI imagery, but there also seem to be some unambiguously artful endeavors (e.g. [1]), no?

[0] https://stephendaitergallery.com/exhibitions/dawoud-bey-pola...

[1] https://www.clairesilver.com/

Re: Curl: We still have not seen a valid security report done with AI help

#242

Post by parliament32 » Wed, May 07, 2025, 3:51 PM UTC

tharant wrote: post #234

namaria wrote: post #225

Earlier quoted context omitted.

In the very narrow fields where I have a deep understanding, LLM output is mostly garbage. It sounds plausible but doesn't stand up to scrutiny. The basics that it can regurgitate from wikipedia sound mostly fine but they are already subtly wrong as soon as they depart from stating very basic facts. Thus I have to assume that for any topic I do not fully understand - which is the vast majority of human knowledge - it…

> In the very narrow fields where I have a deep understanding, LLM output is mostly garbage > Thus I have to assume that for any topic I do not fully understand - which is the vast majority of human knowledge - it is worse than useless, it is actively misleading. Why do you have to make that assumption? An expert arborist likely won’t know much about tuning GC parameters for the JVM but that won’t make them “worse th…

There's a term corollary to what you're trying to argue here: https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

> You open the newspaper to an article on some subject you know well... You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.

Re: Curl: We still have not seen a valid security report done with AI help

#243

Post by LocalH » Wed, May 07, 2025, 4:30 PM UTC

bloppe wrote: post #232

cedws wrote: post #229

Earlier quoted context omitted.

If you charge a fee the motivation for good samaritan reports goes to zero.

That's why they offer cash bounties. You don't need to charge a fee if there is no bounty (aka an actual good Samaritan situation), cuz then there's no incentive to flood it with slop

Another comment in this overall thread indicated that they still receive LLM slop despite not offering bounties. Clout can be as alluring a drug as money.

Re: Curl: We still have not seen a valid security report done with AI help

#244

Post by Vilian » Wed, May 07, 2025, 4:51 PM UTC

LocalH wrote: post #243

bloppe wrote: post #232

Earlier quoted context omitted.

That's why they offer cash bounties. You don't need to charge a fee if there is no bounty (aka an actual good Samaritan situation), cuz then there's no incentive to flood it with slop

Another comment in this overall thread indicated that they still receive LLM slop despite not offering bounties. Clout can be as alluring a drug as money.

Curl has dozen of garbage bug reports made using AI where even the author can't point where the bug if, they answer with "the AI said so it's true"

Re: Curl: We still have not seen a valid security report done with AI help

#245

Post by immibis » Wed, May 07, 2025, 4:54 PM UTC

phs318u wrote: post #213

danielvf wrote: post #101

I handle reports for a one million dollar bug bounty program. AI spam is bad. We've also never had a valid report from an by an LLM (that we could tell). People using them will take any being told why a bug report is not valid, questions, or asks for clarification and run them back through the same confused LLM. The second pass through generates even deeper nonsense. It's making even responding with anything but "clo…

>It's the people that concern me. They cannot tell the difference between truth and garbage. Suffice to say, this statement is an accurate assessment of the current state of many more domains than merely software security.

This has been going for years before AI - they say we live in a "post-truth society". The generation and non-immediate-rejection of AI slop reports could be another manifestation of post-truth rather than a cause of it.

Re: Curl: We still have not seen a valid security report done with AI help

#246

Post by senordevnyc » Wed, May 07, 2025, 5:00 PM UTC

krapp wrote: post #237

senordevnyc wrote: post #236

Earlier quoted context omitted.

You can’t fix AI slop with technology because anything you do to detect it will be incorporated into better models until they evade your tests. How is that a bad thing? At a certain point, it’s no longer AI slop! https://xkcd.com/810/

Polite slop is still slop. Most people use platforms like HN to engage in conversation with other people, not simply to assimilate information as efficiently as possible. That they are conversing with actual human beings has value to them, even when they do human things like express emotions and humor. Hacker News could be perfectly civil if it removed the human element entirely and had an AI post links and generate…

The comic is an example of the principle, not advocating that our ultimate goal is discourse that is civil or polite.

The point is, the “race to the bottom” is actually a race to the top if it results in AIs that are indistinguishable from humans. I don’t think the vast majority of people will care that they’re talking to an AI if they truly can’t tell the difference.

Would you? How do you know I’m not an AI?

Re: Curl: We still have not seen a valid security report done with AI help

#247

Post by cratermoon » Wed, May 07, 2025, 5:16 PM UTC

Seb-C wrote: post #218

danielvf wrote: post #101

I handle reports for a one million dollar bug bounty program. AI spam is bad. We've also never had a valid report from an by an LLM (that we could tell). People using them will take any being told why a bug report is not valid, questions, or asks for clarification and run them back through the same confused LLM. The second pass through generates even deeper nonsense. It's making even responding with anything but "clo…

> I believe that one day there will be great code examining security tools. As for programming, I think that we will simply continue to have incrementally better tools based on sane and appropriate technologies, as we have had forever. What I'm sure about is that no such tool can come out of anything based on natural language, because it's simply the worst possible interface to interact with a computer.

people have been trying various iterations of "natural language programming" since programming languages were a thing. Even COBOL was supposed to be more natural than other languages of the era.

https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

Re: Curl: We still have not seen a valid security report done with AI help

#248

Post by cratermoon » Wed, May 07, 2025, 5:19 PM UTC

billyoneal wrote: post #157

harrisi wrote: post #131

Earlier quoted context omitted.

This is interesting because they've apparently made a couple thousand dollars reporting things to other companies. Is it just a case of a broken clock being right twice a day? Seems like a terrible use of everyone's time and money. I find it hard to believe a random person on the internet using ChatGPT is worth $1000.

There are places that will pay bounties on even very flimsy reports to avoid the press / perception that they aren't responding to researchers. But that's only going to remain as long as a very small number of people are doing this. It's easy for reputational damage to exceed $1'000, but if 1000 people do this...

One might even call it reputational blackmail. "Give me $1000 for this invalid/useless bug report or I'll go to the most click-baity incompetent tech press outlets with how your product is the worst thing since ILUVYOU."

Re: Curl: We still have not seen a valid security report done with AI help

#249

Post by ASalazarMX » Wed, May 07, 2025, 5:23 PM UTC

datatrashfire wrote: post #175

danielvf wrote: post #101

I handle reports for a one million dollar bug bounty program. AI spam is bad. We've also never had a valid report from an by an LLM (that we could tell). People using them will take any being told why a bug report is not valid, questions, or asks for clarification and run them back through the same confused LLM. The second pass through generates even deeper nonsense. It's making even responding with anything but "clo…

> I believe that one day there will be great code examining security tools. Based on current state, what makes you think this is given?

The improvement history of tools beside LLMs, I suspect. First we had syntax highlighting, and we were wondered. Now we have fuzzers and sandbox malware analysis, who knows what the future will bring?

Re: Curl: We still have not seen a valid security report done with AI help

#250

Post by exegete » Wed, May 07, 2025, 5:45 PM UTC

zulban wrote: post #140

joaohaas wrote: post #133

Earlier quoted context omitted.

I unironically can't remember a single case where AI managed to find a vulnerability in an open source project. And most contributions with 'AI help' tend to not follow the code practices of the code base itself, while also in general generating worse code. Also, just like in HTTP stuff 'if curl does it its probably right', I'm also tend to think that 'if the curl team says something its bullshit its probably bullshi…

You wouldn't say "the Google search engine contributed to an open source project". Similarly, many millions of developers are using AI. Sometimes in a good way. When that results in a good MR, they likely don't even mention they used Google, or stackoverflow, or AI, they just submit.

Yes and surely someone somewhere though can be explicit and show they used AI in these cases? It would be nice to curate a list where it has been successful.