Earlier quoted context omitted.
The misalignment came from the model being given an impossible task. A task the required accessing a url. So it got RCE on its own artifactory instance to achieve that. That’s intriguing and worth investigating. If I were them I don’t know if I would have pulled the plug completely at that point either. The introduction of this more advanced “persistent” model that orchestrated an offensive attack against a different…
> The misalignment came from the model being given an impossible task. A task the required accessing a url. So it got RCE on its own artifactory instance to achieve that. So, the way I understand it, it actually was possible. It just required means that the creators of the task didn't predict, and these means have been successfully found and utilized. > My concern is what a misaligned model will do when they’re even…
I don’t disagree that the models task was underdefined. All tasks are. So much in language is implicit. And morality/ethics isn’t something you can write down as an explicit list. That’s what makes the alignment problem so difficult. But we can’t throw our hands up and say, well I guess we can’t align these things. And maybe alignment isn’t the right word - but that’s a semantic debate.