Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl: We still have not seen a valid security report done with AI help

linkedin.com

Post reply on HN
Print view View original

151–160 of 258 posts

Re: Curl: We still have not seen a valid security report done with AI help

#151

Post by hedora »

Szpadel wrote: post #108
jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

I find that only acceptable (only little annoying) when this is some lead in case we're we have no idea what could be the issue, it might help to brainstorm and note that this is not verified information is important. most annoying is when people trust chatgpt more that experts they pay. we had case when our client asked us for some specific optimization, and we told him that it makes no sense, then he asked the othe…

I do this occasionally when it's time sensitive, and I cannot find a reasonable source to read. e.g., "ChatGPT says cut the blue wire, not the red one. I found the bomb schematics it claims say this, but they're paywalled."

If that's all the available information and you're out of time, you may as well cut the blue wire. But, pretty much any other source is automatically more trustworthy.

Re: Curl: We still have not seen a valid security report done with AI help

#152

Post by hedora »

bflesch wrote: post #51
vessenes wrote: post #34

Reading the straw that broke the camel's back commit illustrates the problem really well: https://hackerone.com/reports/3125832 . This shit must be infuriating to dig through. I wonder if reputation systems might work here - you could give anyone who id's with an AML/KYC provider some reputation, enough for two or three reports, let people earn reputation digging through zero rep submissions and give someone like 10,…

there is a reputation system already. according to hackerone reputation system, it is a credible reporter. it's really bad

The vast majority of developers are 10-100x more likely to find a security hole in a random tool than spend time improving their reputation on a bug bounty site that pays That makes it extremely hard to build a reputation system for a site like that. Almost all the accounts are going to be spam, and the highest quality accounts are going to freshly created and take ~ 1 action on the platform.

Re: Curl: We still have not seen a valid security report done with AI help

#153

Post by billyoneal »

hashmush wrote: post #24
jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

As much as I'm also annoyed by that phrase, is it really any different from: - I had to Google it... - According to a StackOverflow answer... - Person X told me about this nice trick... - etc. Stating your sources should surely not be a bad thing, no?

The complaint isn't about stating the source. The complaint is about asking for advice, then ignoring that advice. If one asks how to do something, get a reply, then reply to that reply 'but Google says', that's just as rude.

Re: Curl: We still have not seen a valid security report done with AI help

#154

Post by billyoneal »

jstanley wrote: post #136
jacksnipe wrote: post #130

Earlier quoted context omitted.

In my own experience, the vast majority of people using this phrase ARE using it as a source of authority. People will ask me about things I am an actual expert in, and then when they don’t like my response, hit me with the ol’ “well, I asked chatGPT and it said…”

I think you are misunderstanding them. I also frequently cite ChatGPT, as a way to accurately convey my source, not as a way to claim it as authoritative.

I think you are in the minority of people who use that phrase.

Re: Curl: We still have not seen a valid security report done with AI help

#155

Post by hedora »

j_w wrote: post #121
alabastervlog wrote: post #45

Earlier quoted context omitted.

I had Google's in-search "AI" invent a command line switch that would have been very helpful... if it existed. Complete with usage caveats and warnings! This was like two weeks ago. These things suck.

My favorite is when their in search "AI answer" hallucinates on the Golang standard lib. Always makes me happy to see.

You think that's funny? Try using AI help button in Google's office suite the next time you're trying to track down the right button to press.

Re: Curl: We still have not seen a valid security report done with AI help

#156

Post by Analemma_ »

emushack wrote: post #49
vessenes wrote: post #34

Reading the straw that broke the camel's back commit illustrates the problem really well: https://hackerone.com/reports/3125832 . This shit must be infuriating to dig through. I wonder if reputation systems might work here - you could give anyone who id's with an AML/KYC provider some reputation, enough for two or three reports, let people earn reputation digging through zero rep submissions and give someone like 10,…

Reputation systems for this kind of thing sounds like rubbing some anti-itch cream on bullet wound. I feel like the problem seems to me to be behavior, not a technology issue. Personally I can't imagine how miserable it would be for my hard-earned expertise to be relegated to sifting through SLOP where maybe 1 in hundreds or even thousands of inquiries is worth any time at all. But it also doesn't seem prudent to jus…

> I feel like the problem seems to me to be behavior, not a technology issue.

To be honest, this has been a grimly satisfying outcome of the AI slop debacle. For decades, the general stance of tech has been, “there is no such thing as a behavioral/social problem, we can always fix it with smarter technology”, and AI is taking that opinion and drowning it in a bathtub. You can’t fix AI slop with technology because anything you do to detect it will be incorporated into better models until they evade your tests.

We now have no choice but to acknowledge the social element of these problems, although considering what a shitshow all of Silicon Valley’s efforts at social technology have been up to now, I’m not optimistic this acknowledgement will actually lead anywhere good.

Re: Curl: We still have not seen a valid security report done with AI help

#157

Post by billyoneal »

harrisi wrote: post #131
unsnap_biceps wrote: post #16

For those of you who don't want to click into linked in, https://hackerone.com/reports/3125832 is the latest example of a invalid curl report

This is interesting because they've apparently made a couple thousand dollars reporting things to other companies. Is it just a case of a broken clock being right twice a day? Seems like a terrible use of everyone's time and money. I find it hard to believe a random person on the internet using ChatGPT is worth $1000.

There are places that will pay bounties on even very flimsy reports to avoid the press / perception that they aren't responding to researchers. But that's only going to remain as long as a very small number of people are doing this.

It's easy for reputational damage to exceed $1'000, but if 1000 people do this...

Re: Curl: We still have not seen a valid security report done with AI help

#158

Post by codr7 »

kristopolous wrote: post #144
yoyohello13 wrote: post #25

Earlier quoted context omitted.

Seriously. Being able to look up stuff using AI is not unique. I can do that too. This is kind of the same with any AI gen art. Like I can go generate a bunch of cool images with AI too, why should I give a shit about your random Midjourney output.

Comfyui workflows, fine-tuning models, keeping up with the latest arxiv papers, patching academic code to work with generative stacks, this stuff is grueling. Here's an example https://files.meiobit.com/wp-content/uploads/2024/11/22l0nqm... Being dismissive of AI art is like those people who dismiss electronic music because there's a drum machine. Doing things well still requires an immense amount of skill and exhaus…

Makes even less sense when you put it like that, why not invest that effort into your own skills instead?

Re: Curl: We still have not seen a valid security report done with AI help

#159

Post by kevmo314 »

tharant wrote: post #149
cogman10 wrote: post #60

Earlier quoted context omitted.

I recently had this happen from a senior engineer. What's really frustrating is I TOLD them the issues and how to fix it. Instead of listening to what I told them, they plugged it into GPT and responded with "Oh, interesting this is what GPT says" (Which, spoiler, was similar but lacking from what I'd said). Meaning, instead of listening to a real-life expert in the company telling them how to handle the problem they…

Is it possible that what happened was an impedance mismatch between you and the engineer such that they couldn’t grok what you told them but ChatGPT was able to describe it in a manner they could understand? Real-life experts (myself included, though I don’t claim to be an expert in much) sometimes have difficulty explaining domain-specific concepts to other folks; it’s not a flaw in anyone, folks just have different…

Whenever someone has done that to me, it's clear they didn't read the ChatGPT output either and were sending it to me as some sort of "look someone else thinks you're wrong".

Re: Curl: We still have not seen a valid security report done with AI help

#160

Post by Analemma_ »

zulban wrote: post #142
eestrada wrote: post #117

Earlier quoted context omitted.

And that increase in LLM usage has resulted in an enormous increase of code duplications and code churn in said open source projects. Any benefit from new features implemented by LLMs is being offset by the tech debt caused by duplication and the maintenance burden of constantly reverting bad code (i.e. churn). https://arc.dev/talent-blog/impact-of-ai-on-code/

Yes. The internet has also created a ton of email spam but I wouldn't say "we've never seen a single valid contribution to our project that had internet help". Many millions of developers are using AI. Sometimes in a good way. When that results in a good MR, they likely don't even mention they used Google, or stackoverflow, or AI, they just submit.

I mean, I certainly would say “I’ve never seen a single commercial email that was valid and useful to me as a customer”, and this is entirely because of spam. Any unsolicited email with commercial intent goes instantly, reflexively, to the trash (plus whatever my spam filters prevent me from ever seeing to begin with). This presumably has cost me the opportunity to purchase things I genuinely would’ve found useful, and reduced the effectiveness of well-meaning people doing cold outreach for actually-good products, but spam has left me no choice.

In that sense, it has destroyed actual value as the noise crowds out the signal. AI could easily do the same to, like, all Internet communication.

Post reply on HN