Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl: We still have not seen a valid security report done with AI help

linkedin.com

Post reply on HN
Print view View original

171–180 of 258 posts

Re: Curl: We still have not seen a valid security report done with AI help

#171

Post by mitchellpkt »

terom wrote: post #168
nneonneo wrote: post #118

Earlier quoted context omitted.

Good god did they hallucinate the segmentation fault and the resulting GDB trace too? Given that the diffs don’t even apply and the functions don’t even exist, I guess the answer is yes - in which case, this is truly a new low for AI slop bug reports.

The git commit hashes in the diff are interesting: 1a2b3c4..d4e5f6a I think my wetware pattern-matching brain spots a pattern there.

Excellent catch! I had to go back and take a second look, because I completely missed that the first time.

Re: Curl: We still have not seen a valid security report done with AI help

#172

Post by xnx »

jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

I can see why this would be frustrating, but it's probably a good thing to have people be curious and consult an expert system.

Current systems are definitely flawed (incomplete, biased, or imagined information), but I'd pick the answers provided by Gemini over a random social post, blog page, or influencer every time.

Re: Curl: We still have not seen a valid security report done with AI help

#173

Post by delfinom »

kristopolous wrote: post #164
codr7 wrote: post #158

Earlier quoted context omitted.

Makes even less sense when you put it like that, why not invest that effort into your own skills instead?

It is somebody's own skill. Photographers are not painters. People who do modular synths aren't guitarists. Technical DJing is quite different from tapping on a Spotify app on a smartphone. Just because you've exclusively exposed yourself to crude implementations doesn't mean sophisticated ones don't exist.

But you just missed the point.

People aren't trying to push photographs into painted works displays

People who do modular synths aren't typically trying to sell their music as country/rock/guitar based music.

A 3D modeler of a statue isn't pretending to be a sculpturist.

People pushing AI art are trying to slide it right into "human art" displays. Because they are talentless otherwise.

Re: Curl: We still have not seen a valid security report done with AI help

#174

Post by tharant »

cogman10 wrote: post #60
jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

I recently had this happen from a senior engineer. What's really frustrating is I TOLD them the issues and how to fix it. Instead of listening to what I told them, they plugged it into GPT and responded with "Oh, interesting this is what GPT says" (Which, spoiler, was similar but lacking from what I'd said). Meaning, instead of listening to a real-life expert in the company telling them how to handle the problem they…

> I really fear that a number of engineers are going to us GPT to avoid thinking. They view it as a shortcut to problem solve and it isn't.

How is this sentiment not different from my grandfather’s sentiment that calculators and computers (and probably his grandfather’s view of industrialization) are a shortcut to avoid work? From my perspective most tools are used as a shortcut to avoid work; that’s kinda the while point—to give us room to think about/work on other stuff.

Re: Curl: We still have not seen a valid security report done with AI help

#175

Post by datatrashfire »

danielvf wrote: post #101

I handle reports for a one million dollar bug bounty program. AI spam is bad. We've also never had a valid report from an by an LLM (that we could tell). People using them will take any being told why a bug report is not valid, questions, or asks for clarification and run them back through the same confused LLM. The second pass through generates even deeper nonsense. It's making even responding with anything but "clo…

> I believe that one day there will be great code examining security tools.

Based on current state, what makes you think this is given?

Re: Curl: We still have not seen a valid security report done with AI help

#176

Post by marcosdumay »

zulban wrote: post #142
eestrada wrote: post #117

Earlier quoted context omitted.

And that increase in LLM usage has resulted in an enormous increase of code duplications and code churn in said open source projects. Any benefit from new features implemented by LLMs is being offset by the tech debt caused by duplication and the maintenance burden of constantly reverting bad code (i.e. churn). https://arc.dev/talent-blog/impact-of-ai-on-code/

Yes. The internet has also created a ton of email spam but I wouldn't say "we've never seen a single valid contribution to our project that had internet help". Many millions of developers are using AI. Sometimes in a good way. When that results in a good MR, they likely don't even mention they used Google, or stackoverflow, or AI, they just submit.

If they never got a valid contribution to their project through the internet, yes, they would say exactly that.

They don't say it because the internet provides actual value.

Re: Curl: We still have not seen a valid security report done with AI help

#177

Post by terom »

terom wrote: post #168
nneonneo wrote: post #118

Earlier quoted context omitted.

Good god did they hallucinate the segmentation fault and the resulting GDB trace too? Given that the diffs don’t even apply and the functions don’t even exist, I guess the answer is yes - in which case, this is truly a new low for AI slop bug reports.

The git commit hashes in the diff are interesting: 1a2b3c4..d4e5f6a I think my wetware pattern-matching brain spots a pattern there.

Going a bit further, it seems like there's a grain of truth here, HTTP/2 has a stream priority dependency mechanism [1] and this report [2] from Imperva describes an actual Dependency Cycle DoS in the nghttp implementation.

Unfortunately that's where it seems to end... I'm not that familiar with QUIC and HTTP/2, but I think the closest it gets is that the GitHub repo exists and has a `class QuicConnection` [3]. Beyond that, the QUIC protocol layer doesn't have any concept of exchanging stream priorities [4] and HTTP/2 priorities are something the client sends, not the server? The PoC also mentions HTTP/3 and PRIORITY_UPDATE frames, but those are from the newer RFC 9218 [5] and lack the stream dependencies used in HTTP/2 PRIORITY frames.

I should learn more about HTTP/3!

[1] https://blog.cloudflare.com/adopting-a-new-approach-to-http-...

[2] https://www.imperva.com/docs/imperva_hii_http2.pdf

[3] https://github.com/aiortc/aioquic/blob/218f940467cf25d364890...

[4] https://datatracker.ietf.org/doc/html/rfc9000#name-stream-pr...

[5] https://www.rfc-editor.org/rfc/rfc9218.html#name-the-priorit...

Re: Curl: We still have not seen a valid security report done with AI help

#178

Post by jacksnipe »

38 wrote: post #139
Szpadel wrote: post #108

Earlier quoted context omitted.

I find that only acceptable (only little annoying) when this is some lead in case we're we have no idea what could be the issue, it might help to brainstorm and note that this is not verified information is important. most annoying is when people trust chatgpt more that experts they pay. we had case when our client asked us for some specific optimization, and we told him that it makes no sense, then he asked the othe…

> when this is some lead in case we're we have no idea what could be the issue English please

We’re was autocorrected from where

Re: Curl: We still have not seen a valid security report done with AI help

#179

Post by stevage »

tharant wrote: post #174
cogman10 wrote: post #60

Earlier quoted context omitted.

I recently had this happen from a senior engineer. What's really frustrating is I TOLD them the issues and how to fix it. Instead of listening to what I told them, they plugged it into GPT and responded with "Oh, interesting this is what GPT says" (Which, spoiler, was similar but lacking from what I'd said). Meaning, instead of listening to a real-life expert in the company telling them how to handle the problem they…

> I really fear that a number of engineers are going to us GPT to avoid thinking. They view it as a shortcut to problem solve and it isn't. How is this sentiment not different from my grandfather’s sentiment that calculators and computers (and probably his grandfather’s view of industrialization) are a shortcut to avoid work? From my perspective most tools are used as a shortcut to avoid work; that’s kinda the while…

Did you grandpa think that calculators made engineers worse at their jobs?

Re: Curl: We still have not seen a valid security report done with AI help

#180

Post by jacksnipe »

jstanley wrote: post #136
jacksnipe wrote: post #130

Earlier quoted context omitted.

In my own experience, the vast majority of people using this phrase ARE using it as a source of authority. People will ask me about things I am an actual expert in, and then when they don’t like my response, hit me with the ol’ “well, I asked chatGPT and it said…”

I think you are misunderstanding them. I also frequently cite ChatGPT, as a way to accurately convey my source, not as a way to claim it as authoritative.

I have interrogated it in those cases. I was not misunderstanding.
Post reply on HN