21–30 of 46 posts
So we didn't select just for more accurate information...
Tangent: many in IT and engineering don't work on soft skills. I hope this shows it's not that hard.
That's just how people talk — at least, when they're trying to keep conversation productive. There's a reason the "shit sandwich" is part of professional communication etiquette. If you don't know how insecure the person you're talking to is — and yet productive communication with them is required , and there's no conversational arbiter there to enforce that — then you may as well assume for safety's sake that your c…
That wouldn't be my first guess, or my hundredth. I would tend to assign responsibility for the style to the "helpful, harmless assistant" idea that the major vendors enforce.
What? Perhaps it helps characterize their unreliability, but I'm pretty sure the fact of their unreliability was already pretty well established.
Earlier quoted context omitted.
I'd like to introduce you to some of the most productive Germans on the planet and then we can have a frank discussion of whether or not the airy bullshit that passes for business communication is in fact a booster of productivity.
I wonder what such Germans think of the manners of feudal Japan when watching something like the recent tv show Shogun where it's not about feeling insecure, it's about showing proper respect (according to that culture).
If anyone needs a lesson on how the biases of those making models can cause end user effects, we can point to this as an example that they have likely experienced themselves.
Short of randomness there is no unbiased output possible. Something that reflects the real world will show the prejudice that exists there. A perfectly equitable model is therefore biased against the real world.
Reinforcement learning targeting correct answers has the potential to produce brutally honest responses if correctness if favoured beyond all else, but to train models towards the truth, someone has to decide what the truth is.
Perhaps we could do reinforcement towards a priori truths, that would at least be a path to the comically pedantic AI's that often shows up in science fiction.
For chatbots I think you could go a long way with instruction tuning using a data set designed with a particular attention to tone and perspectives. Individual biases can at least be diluted if you use data selected by a diverse group with broad experiences.
Much like programmer art is generally poor but exists because the person who was there to do it was the programmer. We might need to go beyond implementing programmer sociability.
They also have an increasingly disturbing tendency to end a response with a question. Seems like an over engineered reward in RL to keep the conversation going.
"Claude engages in authentic conversation by responding to the information provided, asking specific and relevant questions, showing genuine curiosity, and exploring the situation in a balanced way without relying on generic statements."
Tangent: many in IT and engineering don't work on soft skills. I hope this shows it's not that hard.
Those who want "soft skills" from their engineers are often looking to place blame. It's easier to blame the engineer who didn't raise concern when things were going off the rails.