Should I be worried?
1. Create a directory.
2. Put a text file in it.
3. Try to move the file into a nonexistent directory.
4. Based on the error message, create the target directory and do the move again.
5. Verify that the move was successful.
Then it stopped running commands and tried to have a conversation with me.
It's probably only a matter of time before someone tunes it to be HackGPT and points a thousand of them at the internet, so yeah, I guess it's time to start freaking out.