991–1000 of 1001 posts
Earlier quoted context omitted.
Why was this test even connected to the public internet? Actually, more importantly—why aren't they saying their next test will be airgapped in light of what happened?
It wasn't. The model discovered and exploited a vulnerability in their package manager proxy to (inferred) move laterally through their internal systems to one with open internet access.
Earlier quoted context omitted.
> do their nonsense to get headlines They know what they're doing. It's a playbook. You write scary stuff in the model card to make it look like legitimate whitepaper rEsEarCh, then drip-feed it to the media outlets who make it a headline story. Fear based marketing is the hot trend of the 2020s. But also, they write literal headlines: https://www.anthropic.com/research/agentic-misalignment
You're confusing PR with marketing. Flaws they find aren't going to convince customers to buy the product. But they need to inform the public of what they doing as it's part of the mission. I haven't seen media outlets picking up on "agentic misalignment". The core of your claim is that it's not a legit research. But that's basically a conspiracy theory. We know for a fact that Anthropic employs some of the best peop…
Have -any- of Anthropic's concerns of imminent disaster passed muster?
Earlier quoted context omitted.
They've been doing blatant, tech, scifi marketing for two years at least. If anything, this is just more sophisticated marketing.
More sophisticated as in paying HF to get involved, and hyping up GLM for something that it may not actually detect?
Isn't this a crime that someone is liable for? What happened is that someone hacked into a computer system without permission. Maybe it wasn't intentional -- sure -- and that would be a factor at sentencing. But it sounds like they've admitted to a crime, and obviously our legal system considers the humans involved to be the liable parties; otherwise everyone would just say "my computer did the hacking" and wouldn't…
My take is that there are at least four potential parties that can all be liable: 1. the creator for the LLM. In particular if neglicence or malice is involved. This can also be someone who did a finetune of an existing model. 2. the inference provider. Remember, a model can do harm just by creating tokens (for example cause someone to run amok or kill herself). Inference providers should do a minimal amount of due d…
Earlier quoted context omitted.
Researcher: hack me Model: I committed a crime Researcher: oh my god
Me to a random person: hack out of a secure environment into another secure environment. Random person: I have no clue or ability to do that.
Fascinating. It's a classic paperclip maximizer situation: under-aligned AI uses ion-cannon to unwrap chocolate bar. I'm both surprised this hasn't already happened and impressed by the capabilities here. Coming up with a 0-day to do this is outrageous. A silly related story is that I run `claude` with full permissions but the prod DB passwords are in a different environment and it has read-only with granular securit…
Earlier quoted context omitted.
IMO they hope to make AI a strongly regulated industry, with OpenAI (and Anthropic) becoming military suppliers with their stronger models, and everything Chinese or open-weight gets banned. The competition from the open models is so strong now that this seems to be the only way to keep both companies afloat, given their dire financials. OpenAI probably hoped that they can achieve market lead and then lower the train…
And even that is backfiring, their partner citing GLM being useful there, and available in just a spin. A ban on open weight models is never going to be enforceable.
Earlier quoted context omitted.
> A ban on open weight models is never going to be enforceable. Just watch them try. Look up those Napster witch-burning trials where they wanted 200k $usd per mp3 downloaded. They will scare everyone into believing that open weight models are illegal and very bad.
But did that kill music downloads? Like that line from The Social Network, "Do you wanna buy a tower records Eduardo?"
Earlier quoted context omitted.
It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.
If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…