Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl: We still have not seen a valid security report done with AI help

linkedin.com

Post reply on HN
Print view View original

31–40 of 258 posts

Re: Curl: We still have not seen a valid security report done with AI help

#31

Post by spiffyk »

hashmush wrote: post #24
jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

As much as I'm also annoyed by that phrase, is it really any different from: - I had to Google it... - According to a StackOverflow answer... - Person X told me about this nice trick... - etc. Stating your sources should surely not be a bad thing, no?

Well, it is not, but the three "sources" you mention are not worth much either, much like ChatGPT.

Re: Curl: We still have not seen a valid security report done with AI help

#32

Post by vel0city »

ivape wrote: post #20
vel0city wrote: post #13

Earlier quoted context omitted.

Clearly the answer is to have an AI assistant handle all the reports from bogus AI auditors. It's all turtles, all the way down.

[flagged]

Cool, so I'll just script something that costs you $0.03/submission but costs me $$0.000001 per submission. We'll see who wins at the end!

> It's an intelligent loop

Highly debatable if most submissions are bogus bullshit.

Re: Curl: We still have not seen a valid security report done with AI help

#33

Post by spiffyk »

parliament32 wrote: post #22

Didn't even have to click through to the report in question to know it would be all hallucinations -- both the original patchfile and the segfault ("ngtcp2_http3_handle_priority_frame".. "There is no function named like this in current ngtcp2 or nghttp3.") I guess these guys don't bother to verify, they just blast out AI slop and hope one of them hits?

> I guess these guys don't bother to verify, they just blast out AI slop and hope one of them hits?

Yes. Unfortunately, some companies seem to pay out the bug bounty without even verifying that the report is actually valid. This can be seen on the "reporter"'s profile: https://hackerone.com/evilginx

Re: Curl: We still have not seen a valid security report done with AI help

#34

Post by vessenes »

Reading the straw that broke the camel's back commit illustrates the problem really well: https://hackerone.com/reports/3125832 . This shit must be infuriating to dig through.

I wonder if reputation systems might work here - you could give anyone who id's with an AML/KYC provider some reputation, enough for two or three reports, let people earn reputation digging through zero rep submissions and give someone like 10,000 reputation for each accurate vulnerability found, and 100s for any accurate promoted vulnerabilities. This would let people interact anonymously if they want to edit, quickly if they found something important and are willing to AML/KYC, and privilege quality people.

Either way, AI is definitely changing economics of this stuff, in this case enshittifying first.

Re: Curl: We still have not seen a valid security report done with AI help

#35

Post by gruez »

spiffyk wrote: post #31
hashmush wrote: post #24

Earlier quoted context omitted.

As much as I'm also annoyed by that phrase, is it really any different from: - I had to Google it... - According to a StackOverflow answer... - Person X told me about this nice trick... - etc. Stating your sources should surely not be a bad thing, no?

Well, it is not, but the three "sources" you mention are not worth much either, much like ChatGPT.

>but the three "sources" you mention are not worth much either, much like ChatGPT.

I don't think I've ever seen anyone lambasted for citing stackoverflow as a source. At best, they chastised for not reading the comments, but nowhere as much pushback as for LLMs.

Re: Curl: We still have not seen a valid security report done with AI help

#36

Post by hx8 »

hashmush wrote: post #24
jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

As much as I'm also annoyed by that phrase, is it really any different from: - I had to Google it... - According to a StackOverflow answer... - Person X told me about this nice trick... - etc. Stating your sources should surely not be a bad thing, no?

It depends on if they are just repeating things without understanding, or if they have understanding. My issue is that people that say "I asked gpt" is that they often do not have any understanding themselves.

Copy and pasting from ChatGPT has the same consequences as copying and pasting from StackOverflow, which is to say you're now on the hook supporting code in production that you don't understand.

Re: Curl: We still have not seen a valid security report done with AI help

#38

Post by tough »

giantg2 wrote: post #27
esafak wrote: post #23

Earlier quoted context omitted.

I had to deal with someone who tried to check in hallucinated code with the defense "I checked it with chatGPT!" If you're just parroting what you read, what is it that you do here?!

Manage people?

then what the fuck are they doing commiting code? leave that to the coders

Re: Curl: We still have not seen a valid security report done with AI help

#39

Post by tough »

hx8 wrote: post #36
hashmush wrote: post #24

Earlier quoted context omitted.

As much as I'm also annoyed by that phrase, is it really any different from: - I had to Google it... - According to a StackOverflow answer... - Person X told me about this nice trick... - etc. Stating your sources should surely not be a bad thing, no?

It depends on if they are just repeating things without understanding, or if they have understanding. My issue is that people that say "I asked gpt" is that they often do not have any understanding themselves. Copy and pasting from ChatGPT has the same consequences as copying and pasting from StackOverflow, which is to say you're now on the hook supporting code in production that you don't understand.

We cannot blame the tools for how they are used by those yielding them.

I can use ChatGPT to teach me and understand a topic or i can use it to give me an answer and not double check and just copy paste.

Just shows off how much you care about the topic at hand, no?

Re: Curl: We still have not seen a valid security report done with AI help

#40

Post by comex »

gruez wrote: post #35
spiffyk wrote: post #31

Earlier quoted context omitted.

Well, it is not, but the three "sources" you mention are not worth much either, much like ChatGPT.

>but the three "sources" you mention are not worth much either, much like ChatGPT. I don't think I've ever seen anyone lambasted for citing stackoverflow as a source. At best, they chastised for not reading the comments, but nowhere as much pushback as for LLMs.

From what I’ve seen, Stack Overflow answers are much more reliable than LLMs.

Also, using Stack Overflow correctly requires more critical thinking. You have to determine whether any given question-and-answer is actually relevant to your problem, rather than just pasting in your code and seeing what the LLM says. Requiring more work is not inherently a good thing, but it does mean that if you’re citing Stack Overflow, you probably have a somewhat better understanding of whatever you’re citing it for than if you cited an LLM.

Post reply on HN