191–200 of 290 posts
Earlier quoted context omitted.
I think it's closer to giving you a diagram of the vanillin molecule and then asking you how many hydrogen atoms you see.
I'm not clear why you think that's closer? The very first thing that happens in most LLMs is that information getting deleted by the letters getting converted into a token stream.
Earlier quoted context omitted.
It’s not like the circumstances of the experiment are significant to the subjects. You’re a college student getting paid $20 to answer questions for an hour. Your response has no bearing on your pay. Who cares what you say?
> Your response has no bearing on your pay. Who cares what you say? Then why not say what you know is right?
We know that LLMs hallucinate, but we can also remove them. I’d love to see a future generation of a model that doesn’t hallucinate on key facts that are peer and expert reviewed. Like the Wikipedia of LLMs https://arxiv.org/pdf/2406.17642 That’s a paper we wrote digging into why LLMs hallucinate and how to fix it. It turns out to be a technical problem with how the LLM is trained.
Earlier quoted context omitted.
It probably shouldn't be called prompt engineering , even informally. The work of an engineer shouldn't require hope .
I don’t think the people who engineered the Golden Gate Bridge, Apollo 7, or the transistor would have succeeded if they didn’t have hope.
Earlier quoted context omitted.
We all use abstractions, and abstractions, good as they are to fight complexity, are also bad because sometimes they hide details we need to know. In other words, we don't genuinely understand anything. We're parrots of abstractions invented elsewhere and not fully grokked. In a company there is no single human who understands everything, it's a patchwork of partial understandings coupled functionally together. Even…
Wholeheartedly agree. Which is why the most valuable people in a company are those who can cross abstraction layers, vertically or horizontally, and reduce information loss from boundaries between abstractions.
Earlier quoted context omitted.
We all use abstractions, and abstractions, good as they are to fight complexity, are also bad because sometimes they hide details we need to know. In other words, we don't genuinely understand anything. We're parrots of abstractions invented elsewhere and not fully grokked. In a company there is no single human who understands everything, it's a patchwork of partial understandings coupled functionally together. Even…
Wholeheartedly agree. Which is why the most valuable people in a company are those who can cross abstraction layers, vertically or horizontally, and reduce information loss from boundaries between abstractions.
Earlier quoted context omitted.
> Probably for the best that users see the words "Sorry, I hallucinated" every now and then. Wouldn’t “sorry, I don’t know how to answer the question” be better?
"Sorry, I just made that up" is more accurate.
Earlier quoted context omitted.
I'm not clear why you think that's closer? The very first thing that happens in most LLMs is that information getting deleted by the letters getting converted into a token stream.
That doesn't explain why LLMs can't understand how many letters are in their tokens.
Ben_wb is pointing out that understanding of concepts is not quite the same as an identical experience of the way they are conveyed. I can use a translation app to to correspond with someone who only knows Mandarin, and they'll understand the concept of vowels, even if they can't tell me how many vowels are in the original sentence I wrote, because that sentence was lost in translation.
KaoD is pointing out that if the system really understands anything nearly as well as it first appears, it should still perform better than it does. My hypothetical Chinese pen-pal would at least be able to recognize and explain why they can't answer me, even if they don't have all the information to solve it.
Earlier quoted context omitted.
It's interesting that they're in the 3rd person - "Claude is", "Claude responds", instead of "you are", "you respond".
Given that it's a big next-word-predictor, I think it has to do with matching the training data. For the vast majority of text out there, someone's personality, goals, etc. are communicated via a narrator describing how thing are. (Plays, stories, almost any kind of retelling or description.) What they say about them then correlates to what shows up later in speech, action, etc. In contrast, it's extremely rare for s…
That was instruction-tuned, RLHFed, system-prompt-priority-tuned, maybe synthetic-data-tuned, and who knows what else.
Maybe they just used illeisms in system prompt prioritization tuning.
Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses…
How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.