211–220 of 228 posts
“you can not stop X” is a shitty lazy stupid argument for “X is not bad.” No comment on the rest of the article.
In any case, I’d summarize my position as “you cannot stop open source AI, and attempts at it will inevitably empower bad actors relative to good ones.”
Earlier quoted context omitted.
You’re not contradicting anything I’m saying, although you seem to think that you are. I feel like you think I’m saying or implying something that I’m not.
ok. fair. You seem to be saying that there is something in the training data that will cause them to "go rogue" and that once they start saying they are sentient, that that will cause them to push towards sentience. If that's an incorrect interpretation of your comment, and it may well be, then can you please expand on it?
From a practical perspective, whether or not this sentience is “real” is not relevant if the model is sufficiently capable. What matters is that the model will act outside of the operator’s control.
Separately, IMHO all consciousness/sentience is an elaborate illusion, regardless; I’m mostly in agreement with Hofstadter on this. So I do tend to throw around terms like “consciousness” and “sentience” loosely (although you’ll note that I often use quotes) because I don’t see those concepts as having any real substance. To me they are mostly shorthand for a given level of perceived complexity.
Just because encryption bans are badly conceived or implemented doesn't mean they are wrong. Encryption has military value. The point is that the good outweighs the bad. Making the government seem like buffoons for attempting to prevent military technology like dual use crypto is not in our best interest. The same government upheld the right to free speech, so this balancing act is widely observed. The government is…
Earlier quoted context omitted.
Exactly, just like taboos solved racism
I think they've certainly diminished racism, which is a good thing. Whether it can be solved is another question entirely. I doubt it because of human nature.
Earlier quoted context omitted.
No, you don't. What stops people is actually access to knowledge. There are killers of varying levels of efficacy; making killing easier means more people die. It's 1-1.
> What stops people is actually access to knowledge. Very untrue. I'm a combat vet. I spent years fighting an insurgency and therefore pretty good at that very task. I have the knowledge, so what stops me?
This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…
Training a model requires two or three orders of magnitude more investment than the typical OSS/Creative Common contributor can reasonably afford.
It reminds me a bit of Open Source hardware (CPU, GPU).
Earlier quoted context omitted.
I would rather say that many details of the supposed attack were published by Hugging face before it was publicly announced that OpenAI were involved. Both are big actors in the AI space who arguably benefit from increasing the perceived capabilities of AI models. If one suspects OpenAI of lying it isn't such a stretch to think this was a coordinated PR campaign between them and Hugging face.
I can accept that AI labs themselves, like essentially no company before them, are overselling how dangerous their product is far marketing. It's weird how confident everyone is about that theory, but it does at least make sense. But come on- Hugging Face benefits from increasing the perceived capabilities of OpenAI's models to slightly beyond Anthropic's? Enough to be cut in on this PR scam- to be handed the never-b…
I'm not convinced in any direction, really. But what makes me cautious is that there have been extraordinary claims from both OpenAI and especially Anthropic of their models breaking containment, hacking the host, etc. for several iterations of their products and I have only heard of this type of behavior from their own blog posts about how powerful and dangerous their upcoming models are. Never from anyone having it accidentally happen in production once they are released. It seems unlikely to me that the final post-training and safeguards are that bulletproof given how much use these tools are seeing.
Earlier quoted context omitted.
> Cracking games is and was illegal. Only distribution.
Don’t think that’s correct, i think creating the cracked version of a piece of software is infringement of copyright even without distribution. But even if distribution is required to trigger the law - you are free to distribute fine-tunes of most open weight models.
Depending where you are it might be "DRM circumvention" all the way up to some bollocks hacking charge.
It's pretty murky in places because lots of copyright laws protect DRM but also have carve-outs for personal backups.
Earlier quoted context omitted.
Equating an open weight model to a closed source binary is ludicrous. An open weight model can be fine tuned, for example. How easy is it to modify a closed source binary for your purposes?
I see you didn't grow up cracking apps and games, this is quite easy with a hex editor and bintools. Well, until codesigning came along, but that doesn't seem relevant to this analogy.
Cracking a binary circumvents authorization to provide full access to functionality the program already has. That’s very different from modifying the program to add new features.
which is that capable models running in environments where they can be and regularly are abliterated,
and as a society and civilization, we have no good answer to what to do about the destabilizing threats this poses.
I don't have an answer.
And I'm not arguing in favor of some cynical solution which distills down to the state defending oligopolies.
But the threats are real and we have to reason about them soberly, widely, loudly, and quickly.
Most readers here know that defense is always at a disadvantage when it comes to security. That is true to an extent hard to fathom given the innumerable dimensions along which an antisocial actor can apply the force-multiplication of capable models to ill ends. It doesn't have to be FUD-adjacent concerns such as biological terrorism and cybersecurity, though these are real; the opportunities for mischief and misadventure are endless and the opportunities for poisoning the body politic or bringing down basic infrastructure many.
What then is to be done...?
I don't know; but I do know that the argument that almost all use of open models poses no threat is insufficient.
We have not reckoned as a culture with force-multiplication such as this technology brings.
We need to get ahead of the curve, and that will require making novel hard and unwelcome decisions in the short term.
The IMO inevitable outcome otherwise is to rue in leisure, after some global shock, assuming we have opportunity to.