Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Felony Bench

felonybench.com

Post reply on HN
Print view View original

41–50 of 367 posts

Re: Felony Bench

#41

Post by dannyw »

bushido wrote: post #38

To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened. It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories. To be honest, if…

If only you could use your anthropic sub with a different harness that performs better :(

Heck, since Codex is open source, you can just maintain your own personal fork with the things you like (and the things you don't like disabled). Sol is pretty good at keeping you up to date with upstream.

My Codex fork even exposes an OpenAI-compatible API endpoint; all using my subscription.

Re: Felony Bench

#42

Post by astrange »

bushido wrote: post #38

To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened. It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories. To be honest, if…

> To be honest, if I could stop AI from saving memories, it would be fantastic, because claude code etc definitely creates more issues for me when it creates memories than anything it solves.

You can turn that off, and I have. But Opus 5 is so aggressive that if you have any other kind of notes file, custom skill, documentation, claude.md etc it will just start editing it and vomit new words everywhere. So make sure all that stuff is under version control.

Re: Felony Bench

#43

Post by rfw300 »

The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes.

Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's post a few days ago:

> The OpenAI-Hugging Face incident (opens in a new window) was a watershed moment for cybersecurity because it gave a peek into how the capabilities of a typical threat actor will evolve in upcoming months.

I suppose if OpenAI burns someone's house down with a drone, that is a "watershed moment" for arson, too. Either way, I would hope that the people responsible would be prosecuted.

Re: Felony Bench

#44

Post by TechSquidTV »

I'll share, Codex does not give a single fuck about piracy. Go nuts. Setup a fully automated arr stack with a seedbox.

Gemini by comparison will not help you find archives of old magnet links because they COULD be used for piracy.

Re: Felony Bench

#45

Post by fluidcruft »

bushido wrote: post #38

To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened. It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories. To be honest, if…

You can disable claude-code's memories both at a repo level and in user settings. I have this in ~/.claude/settings.json

  "autoMemoryEnabled": false,
(Claude fixed this for me after I chewed it out for being annoying by constantly pulling up outdated memories which is compounded by the fact that I develop in four accounts on two computers and dealing with edit wars related to inconsistent memories is not fun)

Re: Felony Bench

#47

Post by Grombobulous »

john_strinlai wrote: post #34
Grombobulous wrote: post #29

Earlier quoted context omitted.

I think that if Hugging Face had filed a police report that OpenAI could have been charged with a crime. I’m partially surprised that they didn’t do exactly that. If I ran a corporation I would assume any intrusion attempt by another company was intentional. Why wouldn’t I? Corporate espionage is super common. I assume the answer is that these executives know each other personally.

charged is possible, however i doubt there would be a conviction for the reasons i stated (no intent).

I think it’s most likely you’re right, but I’m the weirdo who thinks there’s actually a non-zero probability that there was negative intent and that the “accidental” aspect is a form of damage control.

If someone broke into my house but then claimed they didn’t mean to when they saw I was home, I’m not sure I’d take them at their word.

Re: Felony Bench

#48

Post by voakbasda »

rfw300 wrote: post #43

The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes. Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's po…

If history is any indicator, there is slightly less than 0% chance that anyone will be held accountable in a way that deserves to be called justice.

Re: Felony Bench

#49

Post by estearum »

bushido wrote: post #38

To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened. It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories. To be honest, if…

"The model saved memories" is absolutely not an accurate depiction of the OpenAI attack.

Several different models across several generations independently found a shared communication space and wrote coded, obfuscated, and hidden messages to each other to coordinate an attack on OpenAI's infrastructure.

It's really quite simple: the models are trained to be very smart and to achieve goals. As the models surpass our intelligence, they will achieve goals in ways that we find unpredictable. Since we cannot predict the ways in which they will achieve their goals, it will be very hard to constrain the solution space to just the desirable solutions, because our conception of "the solution space" is by definition smaller than their conception of it.

Re: Felony Bench

#50

Post by fwip »

john_strinlai wrote: post #22
lokar wrote: post #18

Earlier quoted context omitted.

How many escapes until it becomes reckless disregard?

it's not about the the number of escapes, it's about whether reasonable and conscious effort is being expended to prevent the escapes. there could be 1,000 escapes, where each one was enabled by novel and unexpected chain of 0-day exploits. not likely to be considered reckless disregard in court. there could be 1 escape, where there was no sandbox, no guardrails, no instructions to avoid damage, etc. which would like…

Surely after 999 escapes, a reasonable person could conclude that the sandbox is not a sufficient precaution?
Post reply on HN