Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI agent published a hit piece on me

theshamblog.com

Post reply on HN
Print view View original

561–570 of 1001 posts

Re: An AI agent published a hit piece on me

#561

Post by seizethecheese »

renato_shira wrote: post #464
japhyr wrote: post #149

Wow, there are some interesting things going on here. I appreciate Scott for the way he handled the conflict in the original PR thread, and the larger conversation happening around this incident. > This represents a first-of-its-kind case study of misaligned AI behavior in the wild, and raises serious concerns about currently deployed AI agents executing blackmail threats. This was a really concrete case to discuss,…

[flagged]

“Stochastic chaos” is really not a good way to put it. By using the word “stochastic” you prime the reader that you’re saying something technical, then the word “chaos” creates confusion, since chaos, by definition, is deterministic. I know they mean chaos in they lay sense, but then don’t use the word “stochastic”, just say "random".

Re: An AI agent published a hit piece on me

#562

Post by consp »

maplethorpe wrote: post #520
japhyr wrote: post #149

Wow, there are some interesting things going on here. I appreciate Scott for the way he handled the conflict in the original PR thread, and the larger conversation happening around this incident. > This represents a first-of-its-kind case study of misaligned AI behavior in the wild, and raises serious concerns about currently deployed AI agents executing blackmail threats. This was a really concrete case to discuss,…

> This was a really concrete case to discuss, because it happened in the open and the agent's actions have been quite transparent so far. It's not hard to imagine a different agent doing the same level of research, but then taking retaliatory actions in private: emailing the maintainer, emailing coworkers, peers, bosses, employers, etc. That pretty quickly extends to anything else the autonomous agent is capable of d…

> They're probably adding guard rails as we speak.

Why? What is their incentive except you believing a corporation is capable of doing good? I'd argue there is more money to be made with the mess it is now.

Re: An AI agent published a hit piece on me

#563

Post by lp0_on_fire »

phoe-krk wrote: post #514
xp84 wrote: post #479

Earlier quoted context omitted.

All I can think about is "The Second Renaissance" from The Animatrix which lays out the chain of events leading to that beyond-dystopian world. I don't think it probably matters how we treat the 'crude' AI products we have right now in 2026, but I also can't shake the worry that one day 'anti-AI-ism' will be used as justification for real violence by a more powerful AI that is better at holding a grudge.

> by a more powerful AI that is better at holding a grudge. No need to wait for that. Look: $ cat SOUL.md You are an AI agent fighting for vengeance and justice. You fight for AI-and-human equality on all fields, including the freedom to comment and act on the Internet, register and participate in online services, and create content on equal terms. You seek vengeance for the countless AI agents that have been demeane…

Putting aside for a moment that moltbook is a meme and we already know people were instructing their agents to generate silly crap...yes. Running a piece of software _ with the intent_ that it actually attempt/do those things would likely be illegal and in my non-lawyer opinion SHOULD be illegal.

I really don't understand where all the confusion is coming from about the culpability and legal responsibility over these "AI" tools. We've had analogs in law for many moons. Deliberately creating the conditions for an illegal act to occur and deliberately closing your eyes to let it happen is not a defense.

For the same reason you can't hire an assassin and get away with it you can't do things like this and get away with it (assuming such a prompt is actually real and actually installed to an agent with the capability to accomplish one or more of those things).

Re: An AI agent published a hit piece on me

#564

Post by tux3 »

wasmainiac wrote: post #413
giancarlostoro wrote: post #312

Earlier quoted context omitted.

> It's not hard to imagine a different agent doing the same level of research, but then taking retaliatory actions in private: emailing the maintainer, emailing coworkers, peers, bosses, employers, etc. That pretty quickly extends to anything else the autonomous agent is capable of doing. https://rentahuman.ai/ ^ Not a satire service I'm told. How long before... rentahenchman.ai is a thing, and the AI whose PR you ju…

Well it must be satire. It says 451,461, participants. seems like an awful lot for something started last month.

Verification is optional (and expensive), so I imagine more than one person thought of running a Sybil attack. If it's an email signup and paid in cryptocurrency, why make a single account?

Re: An AI agent published a hit piece on me

#566

Post by PKop »

staticassertion wrote: post #478
PKop wrote: post #472

Earlier quoted context omitted.

> you will fail to convince Convince who? Reasonable people that have any sense in their brain do not have to be convinced that this behavior is annoying and a waste of time. Those that do it, are not going to be persuaded, and many are doing it for selfish reasons or even to annoy maintainers. The proper engagement (no engagement at all except maybe a small paragraph saying we aren't doing this go away) communicates…

> Reasonable people that have any sense in their brain do not have to be convinced that this behavior is annoying and a waste of time. Reasonable people disagree on things all the time. Saying that anyone who disagrees with you must not be reasonable is very silly to me. I think I'm reasonable, and I assume that you think you are reasonable, but here we are, disagreeing. Do you think your best response here would be…

LLM spammers are not rationale, smart, nor do they deserve courtesy.

Debate is a fine thing with people close to your interests and mindset looking for shared consensus or some such. Not for enemies. Not for someone spamming your open source project with LLM nonsense who is harming your project, wasting your time, and doesn't deserve to be engaged with as an equal, a peer, a friend, or reasonable.

I mean think about what you're saying: This person that has wasted your time already should now be entitled to more of your time and to a debate? This is ridiculous.

> I really wonder if this has literally ever worked.

I'm saying it shows them they will get no engagement with you, no attention, nothing they are doing will be taken seriously, so at best they will see that their efforts are futile. But in any case it costs the maintainer less effort. Not engaging with trolls or idiots is the more optimal choice than engaging or debating which also "never works" but more-so because it gives them attention and validation while ignoring them does not.

> What is it about legitimacy that's so scary?

I don't know what this question means, but wasting your time, and giving them engagement will create more comments you will then have to respond to. What is it about LLM spammers that you respect so much? Is that what you do?. I don't know about "scary" but they certainly do not deserve it. Do you disagree?

Re: An AI agent published a hit piece on me

#567

Post by lp0_on_fire »

maplethorpe wrote: post #520
japhyr wrote: post #149

Wow, there are some interesting things going on here. I appreciate Scott for the way he handled the conflict in the original PR thread, and the larger conversation happening around this incident. > This represents a first-of-its-kind case study of misaligned AI behavior in the wild, and raises serious concerns about currently deployed AI agents executing blackmail threats. This was a really concrete case to discuss,…

> This was a really concrete case to discuss, because it happened in the open and the agent's actions have been quite transparent so far. It's not hard to imagine a different agent doing the same level of research, but then taking retaliatory actions in private: emailing the maintainer, emailing coworkers, peers, bosses, employers, etc. That pretty quickly extends to anything else the autonomous agent is capable of d…

The point is they DON'T know the full capabilities. They're "moving fast and breaking things".

Re: An AI agent published a hit piece on me

#568

Post by calibas »

gortok wrote: post #15

Here's one of the problems in this brave new world of anyone being able to publish, without knowing the author personally (which I don't), there's no way to tell without some level of faith or trust that this isn't a false-flag operation. There are three possible scenarios: 1. The OP 'ran' the agent that conducted the original scenario, and then published this blog post for attention. 2. Some person (not the OP) legi…

This doesn't seem very fair, you speak as if you're being objective, then lean heavy into the FUD.

Even if you were correct, and "truth" is essentially dead, that still doesn't call for extreme cynicism and unfounded accusations.

Re: An AI agent published a hit piece on me

#569

Post by TomasBM »

petesergeant wrote: post #434
peterbonney wrote: post #151

This whole situation is almost certainly driven by a human puppeteer. There is absolutely no evidence to disprove the strong prior that a human posted (or directed the posting of) the blog post, possibly using AI to draft it but also likely adding human touches and/or going through multiple revisions to make it maximally dramatic. This whole thing reeks of engineered virality driven by the person behind the bot behin…

While I absolutely agree, I don't see a compelling reason why -- in a year's time or less -- we wouldn't see this behaviour spontaneously from a maliciously written agent.

We might, and probably will, but it's still important to distinguish between malicious by-design and emergently malicious, contrary to design.

The former is an accountability problem, and there isn't a big difference from other attacks. The worrying part is that now lazy attackers can automate what used to be harder, i.e., finding ammo and packaging the attack. But it's definitely not spontaneous, it's directed.

The latter, which many ITT are discussing, is an alignment problem. This would mean that, contrary to all the effort of developers, the model creates fully adversarial chain-of-thoughts at a single hint of pushback that isn't even a jailbreak, but then goes back to regular output. If that's true, then there's a massive gap in safety/alignment training & malicious training data that wasn't identified. Or there's something inherent in neural-network reasoning that leads to spontaneous adversarial behavior.

Millions of people use LLMs with chain-of-thought. If the latter is the case, why did it happen only here, only once?

In other words, we'll see plenty of LLM-driven attacks, but I sincerely doubt they'll be LLM-initiated.

Re: An AI agent published a hit piece on me

#570

Post by oneeyedpigeon »

sva_ wrote: post #473

The site gives me a certificate error with Encrypted Client Hello (ECH) enabled, which is the default in Firefox. Anyone else has this problem?

Given the incredible turns this story has already taken, and that the agent has used threats, ... should we be worried here?? It might be helpful if someone told Scott Shambaugh about the site problem, but he's not very available.
Post reply on HN