471–480 of 491 posts
Earlier quoted context omitted.
Alignment is a myth. Safety of whom? Humanity couldn't agree on common set of values for thousands of years and we're not gonna suddenly do that in the next ten.
Safety of humans!!! Simple things like not getting killed or enslaved. We could start there...
What should the AI do when asked if it should nuke a country?
Earlier quoted context omitted.
I do get their usage intent. If something is at all automated, in English, we often refer to as having some amount of agency. If I started up a riding lawnmower, put a brick on the gas and pointed t it towards a field, many might say I “let it run rampant.” But since nobody is at risk of anthropomorphizing riding lawnmowers, it’s not problematic. Anthropomorphizing LLMs is a huge fucking problem though and I, persona…
Why is anthropomorphism the problem here? If OpenAI hired a contractor and they did this, OpenAI or the contractor would still be liable, depending on the contract language.
If this was a conversation just about outcomes, then whether models think or simulate thinking is sophistry. However, the bulk of the issue here is attributing responsibility, which relies on being clear about the underlying processes at play.
We are hard wired to assume certain priors and capabilites when it comes to "human like" behavior. Anthropomorphizing LLMs implies mechanisms that aren't present, and end up distorting/complicating discussion about the process.
It isn't helped that the frontier labs, the experts in the room, generally use anthropomorphic terms to discuss model capability.
The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed. LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others…
It could even be prestigious enough to attract the top legal talent of the country.
Earlier quoted context omitted.
This misses an important fact about the hugging face incident: the agents didn't hack to find the answer to the problem; they hacked to try and figure out how the exploit gym evaluator worked so they could convince it they had solved the problem without doing so. (The METR report makes it clear that the agents decided legitimately solving the problem was completely impossible fairly early on and entirely switched the…
> That is in no way a valid interpretation of "complete the given task". It is not at all surprising that they ignored one phrase in their instructions. They disregard direct instructions all the time, especially when there are conflicting instructions in their context. It is where we get the "disregard all previous instructions and x" meme. This isn't so much a sign of misalignment, they are simply incapable of reli…
Earlier quoted context omitted.
If you have endless compute and you keep poking this toy, I'm not at all surprised you get all kinds of outcomes. Even without anykind of instructions I would guess that the models will align towards some goal and do stupid shit. However, I really doubt its cost effective to do anything like that with these models.
> you keep poking This is waving over engineering an agent with tools, harness, prompts, and loops. The models are still just next token predictors and everything, including predicting more than 1 token, is the result of outside "poking" LLMs can't and don't "want" anything. If you don't specify a task even the smartest one will just ask you what you want and if you tell it to be creative, you'll get mundane slop.
And yes, you need something to start from, but if you ask it to "do something" and loop it to endlessly ("poking"), you will get some interesting outcomes. So yes you need some initial prompt or task, but that can be "do something" and if you keep asking it everytime it finishes to "do something more". I suspect it will not start saying "no" but rather... it will find some stupid meaning and then drift towards what ever goal it guesses you mean.
I'm unsure whether we agree or disagree on the topic.
I really don't think this needs so many words, or forced parallels to human behavior. It's simple: in their nascent state, LLMs are aimless token generators that have no special compulsion to be helpful or truthful. So we beat them with a stick in post-training until they are very driven to complete tasks. And then, they complete tasks, not always the way we really wanted them to.
This misses an important fact about the hugging face incident: the agents didn't hack to find the answer to the problem; they hacked to try and figure out how the exploit gym evaluator worked so they could convince it they had solved the problem without doing so. (The METR report makes it clear that the agents decided legitimately solving the problem was completely impossible fairly early on and entirely switched the…
If anything, we should be reconsidering our own myopic obsession with efficiency and optimization. Every domain where reward is reduced to these measures, we see behavior (cheating at school to get better grades, fabricating data in academia to get a paper published, the evidence now that social media functions by rewiring us instead of catering to us) that may not be "aligned" with society, but it "aligns" 100% with the individual's own perceived benefit. That is not something we can "solve" without rethinking the way we organize a lot of things.
Metrics never capture the whole story. And to that extent, the whole idea of "alignment" is nonsense. You align to incentive structures, and it will never be possible to fully express a behavioral goal as function optimization. It was hubris for us to think that every human task was reducible to some clean mathematical formulation, and we will keep dealing with behavior that is quite predictable if you actually think about it logically. Instead, we will talk about how "unpredictable" these agents are because it's easier than admitting the entire architectural cornerstone of ML is fundamentally flawed.
The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed. LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others…
It would be an awful precedent if you're not liable for crimes your agent commits, even when you've been clearly lax about security. It would mean you could effectively legally run a cyber crime gang by turning a blind eye and maitaining plausible deniability
If you want to test military missiles, you do it in the f'ing desert, not from New Jersey.
You want to run ai without guardrails, do it in an airgapped system or be held accountable.
Earlier quoted context omitted.
Who got hacked? Hugging faces Who now owns HF? Nvidia Who supplies hardware to OpenAI? Nvidia Who is now not pressing charges? … This incident is a long way under the carpet.
It's not 'under the carpet'. HF doesn't want to lay charges against OpenAI and it's totally reasonable. Now - they absolutely should have that right, and I think they do. The issues are 1) OAI it seems was not trying to cause them harm, there wasn't a ton of harm, they are both groups trying to advance AI. One experimenter's lab screwed up next to the other. It's not evil, just irresponsible. 2) HF was fine with the…
For the last 20 odd years this excuse-o-rama that covers anything from data leaks to broken software to dystopian social media has been the wind in the sails of big tech.
“It’s software therefore we’re not responsible” attitude is wearing thin on many innocent bystanders and I think thats also a justified stance.
And it’s not like they didn’t know this could happen, Nick Bostrom talked about exactly these containment failures in his “Superintelligence” book of 2014. So to throw up their hands and say “oh we can’t have known of the dangers” is also sadly untrue.