Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Curl: We still have not seen a valid security report done with AI help

linkedin.com

Post reply on HN
Print view View original

121–130 of 258 posts

Re: Curl: We still have not seen a valid security report done with AI help

#121

Post by j_w »

alabastervlog wrote: post #45
indigodaddy wrote: post #29

Earlier quoted context omitted.

Reminds me of when some LLM (might have been Deepseek) told me I could add wasm_mode=True in my FastHTML python code which would allow me to compile it to WebAssembly, when of course there is no such feature in FastHTML. This was even when I had provided it full llms-ctx.txt

I had Google's in-search "AI" invent a command line switch that would have been very helpful... if it existed. Complete with usage caveats and warnings! This was like two weeks ago. These things suck.

My favorite is when their in search "AI answer" hallucinates on the Golang standard lib. Always makes me happy to see.

Re: Curl: We still have not seen a valid security report done with AI help

#122

Post by silversmith »

mentalpiracy wrote: post #86
hashmush wrote: post #24

Earlier quoted context omitted.

As much as I'm also annoyed by that phrase, is it really any different from: - I had to Google it... - According to a StackOverflow answer... - Person X told me about this nice trick... - etc. Stating your sources should surely not be a bad thing, no?

It is not about stating a source, the bad thing is treating chatGPT as an authoritative source like it is a subject matter expert.

But is "I asked chatgpt" assigning any authority to it? I use precisely that sentence as a shorthand for "I didn't know, looked it up in the most convenient way, and it sounded plausible enough to pass on".

Re: Curl: We still have not seen a valid security report done with AI help

#123

Post by giantrobot »

the_snooze wrote: post #107
tough wrote: post #55

Earlier quoted context omitted.

It can consult any sources about any topic, ChatGPT is as good at teaching as the pupil's capabilities to ask the right questions, if you ask me

I like to ask AI systems sports trivia. It's something low-stakes, easy-to-check, and for which there's a ton of good clean data out there. It sucks at sports trivia. It will confidently return information that is straight up wrong [1]. This should be a walk in the park for an LLM, but it fails spectacularly at it. How is this useful for learning at all? [1] https://news.ycombinator.com/item?id=43669364

But just because it's wrong about sports trivia doesn't mean it's wrong about anything else! /s [0]

[0] https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect

Re: Curl: We still have not seen a valid security report done with AI help

#124

Post by floren »

jsheard wrote: post #57
evandrofisico wrote: post #53

Earlier quoted context omitted.

It is supremely annoying when i ask in a group if someone has experience with a tool or system and some idiot copies my question into some LLM and paste the answer. I can use the LLM just like anyone, if i'm asking for EXPERIENCE it is because I want the opinion of a human who actually had to deal with stuff like corner cases.

If it's not worth writing, it's not worth reading.

Reminds me of something I wrote back in 2023: "If you wrote it with an LLM, it wasn't worth writing" https://jfloren.net/b/2023/11/1/0

Re: Curl: We still have not seen a valid security report done with AI help

#125

Post by RadiozRadioz »

jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

There was a brief period of time in the first couple weeks of ChatGPT existing where people did this all the time on Hacker News and were upvoted for it. I take pride in the fact that I thought it was cringeworthy from the start.

Re: Curl: We still have not seen a valid security report done with AI help

#126

Post by palata »

pixl97 wrote: post #58
jsheard wrote: post #57

Earlier quoted context omitted.

If it's not worth writing, it's not worth reading.

I mean, there is a lot of hand written crap to, so even that isn't a good rule.

> If it's not worth writing, it's not worth reading.

It does NOT mean, AT ALL, that if it is worth writing, it is worth reading.

Logic 101?

Re: Curl: We still have not seen a valid security report done with AI help

#127

Post by mystraline »

ryandrake wrote: post #116
multjoy wrote: post #79

Earlier quoted context omitted.

It may well consult any source about the topic, or it may simply make something up. If you don't know anything about the subject area, how do you know if you are asking the right questions?

LLM fans never seem very comfortable answering the question "How do you know it's correct?"

I'm a moderate fan of LLMs.

I will ask for all claims to be backed with cited evidence. And then, I check those.

In other cases, of things like code generation, I ask for a test harness be written in and test.

In some foreign language translation (High German to english), I ask for a sentence to sentence comparison in the syntax of a diff.

Re: Curl: We still have not seen a valid security report done with AI help

#128

Post by idkfasayer »

meindnoch wrote: post #110
pixl97 wrote: post #58

Earlier quoted context omitted.

I mean, there is a lot of hand written crap to, so even that isn't a good rule.

Both statements can be true at the same time, even though they seem to point in different directions. Here's how: 1. *"If it's not worth writing, it's not worth reading"* is a normative or idealistic statement — it sets a standard or value judgment about the quality of writing and reading. It suggests that only writing with value, purpose, or quality should be produced or consumed. 2. *"There is a lot of handwritten…

[dead]

Re: Curl: We still have not seen a valid security report done with AI help

#129

Post by Frost1x »

jacksnipe wrote: post #15

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

I work in a corporate environment as I’m sure many others do. Many executives have it in their head that LLMs are this brand new efficiency gain they can pad profit margins with, so you should be using it for efficiency. There’s a lot of push for that, everywhere where I work.

I see email blasts suggesting I should be using it, I get peers saying I should be using it, I get management suggesting I should use it to cut costs… and there is some truth there but as usual, it depends.

I, like many others, can’t be asked to take on inefficiency in the name of efficiency ontop of currently most efficient ways to do my work. So I too say “ChatGPT said: …” because I dump lots of things into it now. Some things I can’t quickly verify, some things are off, and in general it can produce far more information than I have time to check. Saying “ChatGPT said…” is the current CYA caveat statement around the world of: use this thing but also take liability for it. No, if you practically mandate I use something, the liability falls on you or that thing. If it’s a quick verify I’ll integrate it into knowledge. A lot of things aren’t.

Re: Curl: We still have not seen a valid security report done with AI help

#130

Post by jacksnipe »

silversmith wrote: post #122
mentalpiracy wrote: post #86

Earlier quoted context omitted.

It is not about stating a source, the bad thing is treating chatGPT as an authoritative source like it is a subject matter expert.

But is "I asked chatgpt" assigning any authority to it? I use precisely that sentence as a shorthand for "I didn't know, looked it up in the most convenient way, and it sounded plausible enough to pass on".

In my own experience, the vast majority of people using this phrase ARE using it as a source of authority. People will ask me about things I am an actual expert in, and then when they don’t like my response, hit me with the ol’ “well, I asked chatGPT and it said…”
Post reply on HN