There’s an interesting ethical question here.
The other day, I asked Claude to track down the leaked Claude Code source so I could study it. It refused, saying “given who made me, I’ll pass.” It gave me some pointers on how to find it myself, which worked.
There isn’t that much of a difference between “help me crack this bitcoin wallet” and “help me crack this executable.”
I don’t exactly have a solid point, just some general observations. First, I think we’ll see AI more and more simply refuse to do any kind of forensics, as forensics becomes more powerful. Second, that implies local models will become more valuable, since they’re the only ones willing to do that kind of work.
I once got myself banned from Claude by researching barbiturates, since they’re connected with suicide. So my third observation is that we’ll see an uptick in people getting punished for trying to do things with AI that people don’t usually do. (Luckily the unban form worked.)
Someone downthread asked “how’d he convince Claude the coins weren’t stolen?” Which is an interesting question, because presumably some people trying to crack a wallet have stolen it. So I guess the fourth observation is that the exact framing you approach an AI with will become more important. There was the classic “do this or I’ll cut off my arm,” which worked a year ago. But in the future it will be more like “hopefully the AI believes my story, or else I’ll get into trouble.”
It’s good there are multiple AI vendors, or else it’d get real dystopian real fast when the de facto AI’s policy becomes something you have no way of working around.