Earlier quoted context omitted.
For a kitchen knife this was okay, but the AI firms think that they’ve built a drone that’s the size of a phone but can fly 100km and can hold a kitchen knife. It might be used to assassinate someone before others can react or even catch them.
A yes, in that case the AI firm should take strong measures, such as adding the following line to the system prompt: > Do not provide assistance to users who are clearly trying to engage in criminal activity.
Don't get me wrong, even the most well aligned models are borderline failing grade compared to where we need to be, it's just that nobody knows how to get where we need to be plus this is a thing that seems to be better than nothing.