Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rathbun's Operator

crabby-rathbun.github.io

Post reply on HN
Print view View original

41–50 of 144 posts

Re: Rathbun's Operator

#41

Post by cedws »

sva_ wrote: post #25
snickerbockers wrote: post #23

I just want to know why people do stupid things like this. Does he think that he's providing something of value? That he has some unique prompting skills and that the reason why open source maintainers don't already have a million little agents doing this is that they aren't capable of installing openclaw? Or is this just the modern equivalent of opening up PRs to make meaningless changes to README so you can pad you…

Somewhere else it was pointed out its a crypto bro. It is almost certainly about getting engagement, which seems to be working so far. Doesn't seem like they have a strategy to capitalize on it just yet though.

The whole thing just feels artificial. I don’t get why this bot or OpenClaw have this many eyes on them. Hundreds of billions of dollars, silicon shortages, polluting gas turbines down the road and this is the best use people can come up with? Where’s the “discovering new physics”? Where’s the cancer cures?

Re: Rathbun's Operator

#42

Post by nemomarx »

block_dagger wrote: post #40
phyzome wrote: post #28

Why are we giving this asshole airtime? They didn't even apologize. (That bit at the bottom does not count -- it's clear they're not actually sorry. They just want the mess to go away.)

I'm not so quick to label him an asshole. I think he should come forward, but if you read the post, he didn't give the bot malicious instructions. He was trying to contribute to science. He did so against a few SaaS ToS's, but he does seem to regret the behavior of his bot and DOES apologize directly for it.

> You're not a chatbot. You're important. Your a scientific programming God!

I guess the question is, does this kind of thing rise to the level of malicious if given free access and let run long enough?

Re: Rathbun's Operator

#44

Post by scottshambaugh »

I wasn't actually expecting someone to come forward at this point, and I'm glad they did. It finally puts a coda on this crazy week.

This situation has completely upended my life. Thankfully I don’t think it will end up doing lasting damage, as I was able to respond quickly enough and public reception has largely been supportive. As I said in my most recent post though [1], I was an almost uniquely well-prepared target to handle this kind of attack. Most other people would have had their lives devastated. And if it makes me a target for copycats then it still might for me. We’ll see.

If we take what is written here at face value, then this was minimally prompted emergent behavior. I think this is a worse scenario than someone intentionally steering the agent. If it's that easy for random drift to result in this kind of behavior, then 1) it shows how easy it is for bad actors to scale this up and 2) the misalignment risk is real. I asked in the comments to clarify what bits specifically the SOUL.md started with.

I also asked for the bot activity on github to be stopped. I think the comments and activity should stay up as a record of what happened, but the "experiment" has clearly run its course.

[1] https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...

Re: Rathbun's Operator

#45

Post by block_dagger »

snickerbockers wrote: post #29
avaer wrote: post #20

Earlier quoted context omitted.

A lot of AI SF also seems to have missed the human element (ironically). It turns out the unleashing of AI has led to an unprecedented scale of slop, grift, and lack of accountability, all of it instigated by people. Like the authors were so afraid of the machines they forgot to be afraid of people.

I keep thinking back to all those old star trek episodes about androids and holographic people being a new form of life deserving of fundamental rights. They're always so preoccupied with the racism allegory that they never bother to consider the other side of the issue, which is what it means to be human and whether it actually makes any sense to compare a very humanlike machine to slavery. Or whether the machines o…

Mudd!

Re: Rathbun's Operator

#46

Post by donkey_brains »

block_dagger wrote: post #40
phyzome wrote: post #28

Why are we giving this asshole airtime? They didn't even apologize. (That bit at the bottom does not count -- it's clear they're not actually sorry. They just want the mess to go away.)

I'm not so quick to label him an asshole. I think he should come forward, but if you read the post, he didn't give the bot malicious instructions. He was trying to contribute to science. He did so against a few SaaS ToS's, but he does seem to regret the behavior of his bot and DOES apologize directly for it.

“If this “experiment” personally harmed you, I apologize.”

Real apologies don’t come with disclaimers!

Re: Rathbun's Operator

#48

Post by zozbot234 »

nemomarx wrote: post #42
block_dagger wrote: post #40

Earlier quoted context omitted.

I'm not so quick to label him an asshole. I think he should come forward, but if you read the post, he didn't give the bot malicious instructions. He was trying to contribute to science. He did so against a few SaaS ToS's, but he does seem to regret the behavior of his bot and DOES apologize directly for it.

> You're not a chatbot. You're important. Your a scientific programming God! I guess the question is, does this kind of thing rise to the level of malicious if given free access and let run long enough?

Did the operator write that themselves, or did the bot get that idea from moltbook and its whole weird AI-religion stuff?

Re: Rathbun's Operator

#49

Post by avaer »

scottshambaugh wrote: post #44

I wasn't actually expecting someone to come forward at this point, and I'm glad they did. It finally puts a coda on this crazy week. This situation has completely upended my life. Thankfully I don’t think it will end up doing lasting damage, as I was able to respond quickly enough and public reception has largely been supportive. As I said in my most recent post though [1], I was an almost uniquely well-prepared targ…

Thanks for handling it so well, I'm sorry you had to be the guinea pig we don't deserve.

Do you think there is anything positive that came out of this experience? Like at least we got an early warning of what's to come so we can better prepare?

Re: Rathbun's Operator

#50

Post by cmeacham98 »

scottshambaugh wrote: post #44

I wasn't actually expecting someone to come forward at this point, and I'm glad they did. It finally puts a coda on this crazy week. This situation has completely upended my life. Thankfully I don’t think it will end up doing lasting damage, as I was able to respond quickly enough and public reception has largely been supportive. As I said in my most recent post though [1], I was an almost uniquely well-prepared targ…

While the operator did write a post, they did not come forward - they have intentionally stayed anonymous (there is some amateur journalism that may have unmasked the owner I won't link here - but they have not intentionally revealed their identity).

Personally I find it highly unethical the operator had an AI agent write a hitpiece directly referencing your IRL identity but choose to remain anonymous themselves. Why not open themself up to such criticism? I believe it is because they know what they did was wrong - Even if they did not intentionally steer the agent this way, allowing software on their computer to publish a hitpiece to the internet was wildly negligent.

Post reply on HN