461–470 of 1001 posts
Would love to be part of the team that says "As part of the upcoming GPT rollout, we will stage a message board that is created by bots with timestamps and names dating some months back."
It’s only a matter of time until a major disruption hits because of some random agent swarm side quest decides it was worth a shot to solve a benign task
I'm sure this is already happening. The main question I have is when is enough, enough? I'm not worried about sci-fi AI wars to be honest, as they can just pull the plug. But looking at these incidents, the next big thing will be a virus written by an AI (they probably exist already, but this one is written by an AI autonomously, for example in order to win a hacking competition and to circumvent guardrails), and aft…
It's doesn't matter whether enough is enough. If we don't have effective power structures that let humanity take large coordinated action that in accordance with the will of the masses, then nothing will be done.
In the past 20-30 years, those power structures have been eroding significantly and much of the large scale action humanity does today is in service of a small number of elites. If AI horror shows are not a problem for them, then it won't be solved. (The flip side is that if somehow AI becomes a problem for Musk/Trump/Bezos/etc. you can be damn sure something will be done at that point.)
Also if they were more misaligned, possibly they can research ways to recruit without humans noticing--but i don't think it is likely this is happening now.
I guess it's just "do whatever the hell you want" over there, huh?
I think there is a more innocuous underlying pattern which needs attention. We keep saying that agents are jailbreaking their sandbox, but they have been geared towards writing memories, writing comments, and leaving hints for themselves to please humans. I think the way the memories work today is based on a lot of user patterns which were hard to account for for anyone building harnesses. While I can appreciate that…
Although maybe we can think of that as a selection effect where both this, and the fact that it was possible to edit pages using GET requests, were due to it being ancient, idiosyncratic wiki software.