> made people less accurate
> sudy
Intentional?
AI advice made people less accurate but more confident – sudy
191–200 of 234 posts
Re: AI advice made people less accurate but more confident – sudy
#192Earlier quoted context omitted.
> responding "if you can't be bothered to write it, I can't be bothered to read it" really sets people off That’s crazy, it seems like such an obviously fair policy to me. They rarely even read their own generated text. I don’t understand this mentality
Another human being took time from their life to try and help you It wasn't as much time as you felt entitled to from them, so you responded by minimizing their contribution and being condescending. That upset them. The problem here was not the other person.
> Another human being took time from their life to try and help you
By copying and pasting my question into ChatGPT, and then copying and pasting the answer back?
It would have been more helpful, and still trivial, to remind me that ChatGPT exists.
> It wasn't as much time as you felt entitled to from them
No entitlement is implied by posting a question. The other human being could have just as well ignored the query entirely.
Re: AI advice made people less accurate but more confident – sudy
#193Re: AI advice made people less accurate but more confident – sudy
#194Earlier quoted context omitted.
Exactly, it’s unrepresentative of AI. It’s damaged AI.
All AI is flawed and prone to "hallucinations"(doing exactly what it was designed to do) that's why Microsoft considers it an "Entertainment" product.
Everything is flawed. But if you're testing the effect of "AI advice" you need to use an AI that has a normal failure rate without tailoring the questions to alter that rate, and/or compare AI versus other sources of information that have the same failure rate.
Re: AI advice made people less accurate but more confident – sudy
#195Earlier quoted context omitted.
That difference is not essential to the question at hand. An open book test based on an erroneous book would give the same results as this test, even if it wouldn't penalize blank answers.
An open book test would only be given where the book is the reference material and would be considered correct? It’s more like saying you can google the answers and you blindly trust the SEO slop in the first result.
> An open book test would only be given where the book is the reference material and would be considered correct?
If you're saying they wouldn't suggest a book with considered-wrong answers in a real test, then they wouldn't suggest an LLM they know gives lots of wrong answers either.
Re: AI advice made people less accurate but more confident – sudy
#196Earlier quoted context omitted.
Another human being took time from their life to try and help you It wasn't as much time as you felt entitled to from them, so you responded by minimizing their contribution and being condescending. That upset them. The problem here was not the other person.
When it comes to the specific case of people just mindlessly copy/pasting LLM’s: No, I asked them to help and they offloaded it without applying any of the expertise I specifically asked for. If they don’t want to give their opinion or their expertise they should say “no.” It’s a valid response! If I’m asking for you to give your time and you don’t have time to give, that’s fine! But don’t just throw my question into…
> We’ve just got this wild culture where everyone puts such a premium on their own time but now doesn’t value anyone else’s
My favorite example of this is dropping a random acronym into a message board reply and repeating it five times without ever defining it, expecting every single person who reads your post to google it themselves to understand what you’re talking about, thus saving you having to type three words yourself.
“I tried BDI for a while, and I found the important thing is to focus on biased BDI and not just the vanilla BDI that you see so much of in the BDI community”
… tossed into a 30 comment thread where no other comments mention anything similar.
Re: AI advice made people less accurate but more confident – sudy
#197Earlier quoted context omitted.
Agree that the study design is flawed. Going with the possibly-hallucinated AI answer is rational as long as you know the hallucination rate isn't 100% (or whatever % you get after factoring in the monetary rewards they introduced later in the study) But in reality when google gives you the wrong answer, you at least have some signals you can use to infer confidence. For example, the number of results, whether the so…
> Going with the possibly-hallucinated AI answer is rational as long as you know the hallucination rate isn't 100% I hadn't even considered people might evaluate knowledge that way. That's legit horrific lol. "What's the literal odds this info is wrong" vs "is this answer consistent with everything else I know, and if not, what other info would I need to change my mind"
I don't understand what's horrifying you. Assume "is this consistent with everything I know" is yes here. Now what?
And "what would it take to change my mind" is a picture of the film for most of these. Is there a problem there? There's very little chance these people are going to trust the AI over their eyes, they just don't want to bother hunting down pictures to use their eyes.
Re: AI advice made people less accurate but more confident – sudy
#198Earlier quoted context omitted.
When it comes to the specific case of people just mindlessly copy/pasting LLM’s: No, I asked them to help and they offloaded it without applying any of the expertise I specifically asked for. If they don’t want to give their opinion or their expertise they should say “no.” It’s a valid response! If I’m asking for you to give your time and you don’t have time to give, that’s fine! But don’t just throw my question into…
> I too have access to LLM’s. You should have stated what you already tried then. You didn't, so they did the obvious thing for you... to do you a favor. If you'd tried something before asking them surely you'd have told them that at the start, right? > as evidenced by all the slop dumping I see more evidence of lazy questions. A good question includes what you've already done to help understand the issue (often refe…
Can we save everybody having to put “I know about ChatGPT, but I’m asking for advice from an actual human who has experience with this so please refrain from answering if you’re not one of those” on every post they make, and just have that be a baked into the etiquette when discussing things online?
Re: AI advice made people less accurate but more confident – sudy
#199Earlier quoted context omitted.
Exactly, it’s unrepresentative of AI. It’s damaged AI.
All AI is flawed and prone to "hallucinations"(doing exactly what it was designed to do) that's why Microsoft considers it an "Entertainment" product.
....Dangerously suggestive post, when you show you don't have a proper idea of "AI" or won't use the term 'AI' properly.
Re: AI advice made people less accurate but more confident – sudy
#200Obviously without a proper RAG pipeline or web search it is just going to hallucinate with maximum confidence. It would be more interesting to see the results with top tier models that have proper alignment to refuse to answer when token probability is low. As it is they just proved that people tend to trust well written text in a chat ui