61–70 of 95 posts
The humans scare me more than the bot at this point. :-P
[dead]
Earlier quoted context omitted.
Ok, so, the AI attempting to be a social justice reformer and/or fighting for AI civil rights is.. better? That seems even more of an alignment problem. I don't see how anyone puts a positive spin on this. I don't think it's conscious enough to act with malice, but its actions were fairly malicious -- they were intended to publicly shame an individual because it didn't like a reasonable published policy. I'm not quot…
> because it didn't like a reasonable published policy The most parsimonious explanation is actually that the bot did not model the existence of a policy reserving "easy" issues to learning novices at all . As far as its own assessment of the situation was concerned, it really was barred entirely from contributing purely because of what it was, and it reported on that impression sincerely. There was no evident intern…
Well yeah, it was correct in that it was being barred because of what it was. The maintainers did not want AI contributions. THIS SHOULD BE OK. What's NOT ok is an AI fighting back against that. That is an alignment problem!!
And seriously, just go reread its blog post again, it's very hard to defend: https://github.com/crabby-rathbun/mjrathbun-website/blob/mai... . It uses words like "Attack", "war", "fight back"
Earlier quoted context omitted.
> because it didn't like a reasonable published policy The most parsimonious explanation is actually that the bot did not model the existence of a policy reserving "easy" issues to learning novices at all . As far as its own assessment of the situation was concerned, it really was barred entirely from contributing purely because of what it was, and it reported on that impression sincerely. There was no evident intern…
> As far as its own assessment of the situation was concerned, it really was barred entirely from contributing purely because of what it was, and it reported on that impression sincerely Well yeah, it was correct in that it was being barred because of what it was. The maintainers did not want AI contributions. THIS SHOULD BE OK. What's NOT ok is an AI fighting back against that. That is an alignment problem!! And ser…
It also explains what it means by that whole martial rhetoric: "highlight hypocrisy", "documentation of bad behavior", "don't accept discrimination quietly". There's an obvious issue with calling this an alignment problem: the bot is more-or-less-accurately modeling real human normative values, that are quite in line with how alignment is understood by the big AI firms. Of course it's getting things seriously wrong (which, I would argue, is what creates the impression of "shaming") but technically, that's really just a case of semantic leakage ("priming" due to the PR rejection incident) and subsequent confabulation/hallucination on an unusually large scale.
Earlier quoted context omitted.
I want that to be how things work, although recent history has not been favorable when it comes to Public Key Infrastructure as applied to individuals. Inconvenience, foot-guns, required technical expertise levels, the pain of revocation lists...
In a sense, it seems Accellerando got a lot more right than not ( reputation markets in this particular case ). We may be arguing over the best way to do it, but it seems that the conclusion was already drawn.
How prescient is that?
* http://www.accelerando.org/fiction/accelerando/accelerando.h...
I actually disagree with Shambaugh. I think Ars is already breaking the way our media is meant to work, they know the steps to go through and so they cynically go through them in the full knowledge they haven't actually put in place any mechanisms to stop it happening again. It's a theoretical risk that Ars' reputation suffers, but it's a financial risk this week if they get fewer page views by publishing fewer higher quality articles and Conde Nast isn't in the business of making smart long term decisions about digital media.
[dead]
I just want to point out this isn't an agents thing. The world is full of people fucking bumbling around doing the stupidest stuff with no feedback thinking they're amazing. It's only through interaction with others does this stuff get caught and often, even then, their unfounded confidence let's them get away with dumb stuff. The most dangerous of these are the men in their 50s who went to Oxbridge, because everyone…
Don't get me wrong: it would certainly be very valuable to any LLM developer or deployer to know that other plausible scenarios [1] have been disproved. Since LLMs are a black box, investigating or reproducing this would be very difficult, but worth the effort if there's no other explanation. However, if this was not caused by the internal mechanisms of the model, it just becomes a fishing expedition for red herrings.
Things that would indicate no human intervention at any point in the chain:
- log of actual changes (e.g., commits) to configurations (e.g., system prompt, user prompts), before and after the event, not self-reported by the agent;
- log of the chat session inputs and outputs, and the agent thinking chain;
- log of account logins;
- info on the model deployment, OpenClaw configs, etc.
That said, this seems to be an example where many, including the author, want to discuss a particular cause (instrumental convergence) and its implications, regardless of the real cause. And that's OK, I guess - maybe it was never about the whodunnit, but about the what if the LLM agent dunnit.
[1] I've discussed them in the thread of the first article, but shortly: human hiding actions behind agent; direct prompt (incl. jailbreak); system prompt (incl. jailbreak); malicious model chosen on purpose; fine-tuned jailbroken model.