141–150 of 234 posts
https://www.youtube.com/watch?v=axOcn--n_lM
https://www.anthropic.com/research/AI-assistance-coding-skil...
We should remember LLM do have legitimate use-cases like search, as we enter the "Trough of disillusionment" in the hype cycle. =3
Earlier quoted context omitted.
> responding "if you can't be bothered to write it, I can't be bothered to read it" really sets people off That’s crazy, it seems like such an obviously fair policy to me. They rarely even read their own generated text. I don’t understand this mentality
Another human being took time from their life to try and help you It wasn't as much time as you felt entitled to from them, so you responded by minimizing their contribution and being condescending. That upset them. The problem here was not the other person.
Earlier quoted context omitted.
I don’t think this is correct, as we’re talking about all online discourse being influenced by AI, and you don’t know what to trust anymore, rather than generating AI output yourself (which is what you are referring to).
In this context, it would mean building a knowledge base the model can link to that is considered authoritative. So maybe the LLM is has some slop in it's training, and that causes it to try and output some junk, but if it can't match it to it's verified ground-truth KB it doesn't end up outputting it. The problem is figuring out what is the authoritative sources.
This could serve as a beacon for those who still care: Richard Feynman: "I have the advantage of having found out how hard it is to get to really know something, how careful you have to be about checking the experiments, how easy it is to make mistakes and fool yourself..." https://www.youtube.com/watch?v=tWr39Q9vBgo
Earlier quoted context omitted.
Agree that the study design is flawed. Going with the possibly-hallucinated AI answer is rational as long as you know the hallucination rate isn't 100% (or whatever % you get after factoring in the monetary rewards they introduced later in the study) But in reality when google gives you the wrong answer, you at least have some signals you can use to infer confidence. For example, the number of results, whether the so…
> Going with the possibly-hallucinated AI answer is rational as long as you know the hallucination rate isn't 100% I hadn't even considered people might evaluate knowledge that way. That's legit horrific lol. "What's the literal odds this info is wrong" vs "is this answer consistent with everything else I know, and if not, what other info would I need to change my mind"
Your alternative is also something that people do, but rarely consciously.
This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…
Why are textbooks relevant here? Even if you repeated the experiment with a textbook instead of the AI and got the same result, what conclusion would you draw from this? The general conclusion of the study seems to be "giving people access to authoritative-seeming but wrong tools for answering questions outside their area of expertise reduces their ability to say they don't know the answer, even when the answer is wr…
Anyone trusting AI as the single authoritative source of information is stupid - but this follows from the fact that trusting anyone as a "singular point" as a source of information is stupid. You corroborate, you intervene on the world to test your mental model, you discuss with other people. That's what learning is. I've never learned from start to back to a textbook before as the single source of information (besides one philosophy of science textbook; in which I spent a month digging around adjacent fields, and then it just so happened that that one textbook synthesized every piece of information I looked up, and it was mostly a consolidating review).
If your study pre-supposes certain courses of action and artificially constrains the action space for the sake of "reproducibility", you may get a result, and a "scientifically rigorous one". But it's not going to say anything about reality in any meaningful way. While anecdotes and the complexity of real life isn't "science" (in that it's a controlled, repeatable, interventional experiment that's subject to a community of critics who want to hold you up to standards of rigor), there's far more truth in how people actually proceed and engage with these tools.
This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…
How often do managers just regurgitate ai advice rather than consulting their experts? How often does a person question an expert because the ai said so?
Naturally, the ai will be right some of the time - but it’s really hard to correct for the times the ai is wrong.
This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…
Agreed, the headline says "AI advice made people three times less accurate". But if we really want to know how accurate these people were, we need to know how accurate the AI system they use is. If the AI system is hobbled to a point where it is worse than they reasonably expect we can't blame the people or the AI system. This would be the same as claiming the listening to experts make people less accurate in a study…
Earlier quoted context omitted.
The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.
> The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. It's not just fanboys doing that, but "official" educational channels. Both an internal training (more like an all-hands knowledge share) and some dumb training session by a guy from Anthropic explicitly, repeatedly recommended users anthropomorphize their chatbots on no uncertain terms. "Treat it as your super-s…
This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…
That strikes me as an incredibly appropriate test because LLM’s are unreliable with factual statements. People need to be able to understand that and not treat them like textbooks which are basically 99.9% accurate (let’s please not bicker over the 99.9%. It’s close enough. A major textbook is safe to treat as accurate, an LLM is not).