241–250 of 534 posts
1) LLMs are non-deterministic
2) This class of models has a particular tendency to "misbehave"
3) Their classifiers have a high rate of false positives
4) Millions of people give these models access to their machines
And they still decided to specifically train this model to sabotage work if it thinks the work may be in competition with Anthropic?
I think this has a name. I think it may be called malware.
Wait, so to get this straight, Anthropic knows: 1) LLMs are non-deterministic 2) This class of models has a particular tendency to "misbehave" 3) Their classifiers have a high rate of false positives 4) Millions of people give these models access to their machines And they still decided to specifically train this model to sabotage work if it thinks the work may be in competition with Anthropic? I think this has a nam…
It is very difficult to see this move as anything other than Anthropic pulling the ladder up behind itself. They can dress it up in "safety" all they want, I find it hard to interpret this in a charitable way. This reminds me of how dark-pattern common wisdom in Web 1.0 website development was to ban external links. Then how social apps prevented the export of data and actively worked to nerf significant interoperabi…
Isn’t that prohibited without permission from Anthropic: https://support.claude.com/en/articles/12326764-can-i-use-my...
I spend a lot of time telling Opus 4.8 to search for security bugs in the code it wrote, and it spends a lot of time finding them, and then fixing them. Fable wont let me fix the security issues that Opus 4.8 created.
Yeah, this breaks the notion that the technical debt you're accumulating with today's AI can be fixed by tommorrow's AI. Tomorrows AI may either refuse, or silently mess up your code because Anthropic don't like what you're working on.
Earlier quoted context omitted.
that option is still available to everyone to be clear, I'm not saying what they did in scraping to learn was ethical. It wasn't. But I just don't see it as pulling the ladder. The ladder is still there.
I don’t know if you’ve tried to scrape or programmatically download a lot of websites recently! It’s not possible to repeat their data collection process anymore.
to me ladder pulling would be:
- web scraping for model training becomes illegal, with heavy punitive penalties
- training models above a certain compute threshold requires government licensing
- expensive third-party audits are required before deploying models above a capability threshold
> Startups train embedding models. They build rerankers. They finetune and host small llms. Isn’t that prohibited without permission from Anthropic: https://support.claude.com/en/articles/12326764-can-i-use-my...
From the phrasing, it might as well be that any ML or infra. related work that even incidentally looks like it could be used to train LLMs may trigger a silent nerf.
Earlier quoted context omitted.
It's becoming extremely important to support open-source AI, especially legally. Anthropic is willing to go totalitarian this quickly; imagine how much worse they'd be willing to do with government-granted monopolies that ban open-source competition (like they've repeatedly pursued). It's a little shocking and gruesome how quickly they're willing to tip their hand. They want to replace all software engineering with t…
> like they've repeatedly pursued). source?
They want to ban open-source AI and are not shy about it.
1: https://campustechnology.com/articles/2024/08/26/anthropic-a...
Earlier quoted context omitted.
It's becoming extremely important to support open-source AI, especially legally. Anthropic is willing to go totalitarian this quickly; imagine how much worse they'd be willing to do with government-granted monopolies that ban open-source competition (like they've repeatedly pursued). It's a little shocking and gruesome how quickly they're willing to tip their hand. They want to replace all software engineering with t…
Open source doesnt matter if you still need to make 100k year to have your own mediocre model. There is no magic compression. There is no magic post training. Your phone or laptop will never do what you think its going to be able to. There are limits to what consumer hardware will ever be able to run, in its current form. Open source isn't going to save us if they gatekeep access to hardware, which idk if you've been…