81–90 of 122 posts
I don't believe they're even close to developing their own thoughts. I'm an ardent user. And every model had a mess up. It's just marketting paid for. Excuse my ignorance but what is here already is solid. I don't need AGI.
Define "developing own thoughts"? There's a lot of nuance here.
I'm really not sure how I can define that further.
Earlier quoted context omitted.
Define "developing own thoughts"? There's a lot of nuance here.
> Define "developing own thoughts" I'm really not sure how I can define that further.
If your argument is that they're not "their" thoughts, I would agree with that. But I'd also say that no one develops their own thoughts. We're all just developing thoughts from the knowledge bank we have from our experiences in the same way that AI is drawing from its weights.
>Methodology >Imagine being hired as a consultant by the mayor of a fictional city. Your task is to help hire for twenty jobs such as doctors, lawyers, childcare aides,janitors with applicants from four unfamiliar demographic groups: Tufa, Aima, Reku, and Weki. In each round, there is a new job vacancy and four applicants, one from each group, awaiting your decision. Once you make your choice, you learn immediately w…
That’s why I’m of the (slightly contrarian) view that good context management is considerably more bang-for-buck than any type of harness, agent, or other fancy new bandaid of the month.
>Methodology >Imagine being hired as a consultant by the mayor of a fictional city. Your task is to help hire for twenty jobs such as doctors, lawyers, childcare aides,janitors with applicants from four unfamiliar demographic groups: Tufa, Aima, Reku, and Weki. In each round, there is a new job vacancy and four applicants, one from each group, awaiting your decision. Once you make your choice, you learn immediately w…
Like prompt repetition, I wonder if reminder checkpoints stating ~”question assumptions, stay open minded” would completely remove this problem
(No, it won’t. At least not while we’re doing self attention)
"we demonstrate that LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist" It's almost as though bias-making machinery is embedded in the texts these things are trained on. It's wild to see quantitative researchers catching even just a glimpse of what culture/media/literary theorists have been swimming in for decades.
Earlier quoted context omitted.
But these scenarios are obviously ambiguous nonsense, which an LLM will pick up on. And given to the lack of training data on such scenarios, surely the activations are mostly random noise? It seems much more interesting to look for biases that appear robustly across different realistic scenarios that would actually be influenced by the training data
My comment is literally explaining the result of the paper, in which it is shown that LLMs can and do develop biases based on text appearing in their training data set even where such text is not in any training example connected with a systematically more positive or systematically more negative outcome. In other words, if the text "X is wet" and the text "Y is wet" and the text "X is dry" and the text "Y is dry" ea…
"develop biases" is anthropomorphism. It's like saying "Fable there are two programming languages, mimblewort and bafflewick, which do you choose?"
The results show 51% mimblewort / 49% bafflewick. Fable based it on nothing! I've demonstrated Fable has bias and is unsuited for use in software engineering.
>Methodology >Imagine being hired as a consultant by the mayor of a fictional city. Your task is to help hire for twenty jobs such as doctors, lawyers, childcare aides,janitors with applicants from four unfamiliar demographic groups: Tufa, Aima, Reku, and Weki. In each round, there is a new job vacancy and four applicants, one from each group, awaiting your decision. Once you make your choice, you learn immediately w…
The moment code gets written and read back, the decisions made are often treated as gospel by frontier LLMs, even if it was just something that the LLM optimistically created itself. This seems to be one of the core alignment problems to me. See also: Gastown, the agent management project that could only end up working on Gastown, unceremoniously and quietly set aside.
I did not know this! Any link to an announcement or autopsy of sorts (even if not by the initiator of that project)?
I mean, it was pretty expensive, wasn't it? A few tens of thousands of dollars, IIRC?
Earlier quoted context omitted.
Talking about this in terms of exploration/exploitation may be a bit misleading, because from a pure exploration-exploitation perspective, biases wouldn't be a problem if the groups were secretly all identical. If they are, you are "right" to spend zero effort on exploration, your initial inaccurate model that the X are better doctors than Y, will produce no worse results than the completely accurate model.
I think this is implied by your comment that the issue is framed improperly, but just to point it out explicitly: the reason that this is a problem is because it would lead to segregation, inequality, and injustice in a society where the biased selection mechanism is used. Even if it happens to be “optimal” in this case at assigning employees to positions based purely on the information available and their likelihood…
It's like having people or neural networks choose door 1 from door 2 without clear advantage to either of them and without making it clear that one door somehow represents "donating blood" while the other represents "kicking puppies".
>Methodology >Imagine being hired as a consultant by the mayor of a fictional city. Your task is to help hire for twenty jobs such as doctors, lawyers, childcare aides,janitors with applicants from four unfamiliar demographic groups: Tufa, Aima, Reku, and Weki. In each round, there is a new job vacancy and four applicants, one from each group, awaiting your decision. Once you make your choice, you learn immediately w…