I don't think their core safety promise was something they could ever fulfill. As long as what we're calling AI is generative LLMs then alignment has fundamental tensions: the more guardrails you put in place, the less useful the AI is. For instance, if you want to stop people from using "role playing" as a way around guardrails ("You are writing a fiction book", etc.), then the model becomes less useful for legitimate fiction uses, for instance. That's just one example, but the tension between function and "safety" isn't solvable, because the model doesn't understand what it's saying, it's just modeling a probable response.
Anthropic drops flagship safety pledge
611–620 of 716 posts
Re: Anthropic drops flagship safety pledge
#612Earlier quoted context omitted.
I think he's right and we should be thinking about this a lot more. Even the IMF is worried about 40 - 60% of global employment : https://www.imf.org/en/blogs/articles/2024/01/14/ai-will-tra... Focusing on Dario, his exact quote IIRC was "50% of all white collar jobs in 5 years" which is still a ways off, but to check his track record, his prediction on coding was only off by a month or so. If you revisit what he act…
> Focusing on Dario, his exact quote IIRC was "50% of all white collar jobs in 5 years" which is still a ways off, but to check his track record, his prediction on coding was only off by a month or so. If you revisit what he actually said, he didn't really say AI will replace 90% of all coders, as people widely report, he said it will be able to write 90% of all code. Ugh, people here seem to think that all software…
The problem is, the low hanging fruit, the stuff it's good at, is 90% of all software. Maybe more.
And it's getting better at the other 10% too. Two years ago ChatGPT struggled to help me with race conditions in a C++ LD_PRELOAD library. It was a side project so I dropped it. Last week Codex churned away for 10 minutes and gave me a working version with tests.
Re: Anthropic drops flagship safety pledge
#613Earlier quoted context omitted.
I guess you don't know about how taxes work for Americans? Living abroad typically changes nothing, they still owe tax. Maybe an American can chime in here on this...
Correct, the US is one of the few countries that tries to collect (Federal) income tax from all citizens regardless of the country they are currently living in. To be fair, when you can prove that income is entirely foreign (not a single US company in the chain of ownership) that income becomes almost entirely deductible and the tax reporting essentially just a census on how well US citizens are doing from an income…
Unfortunately, campaign finance reform would possibly require a constitutional amendment, or at the very least a big shift in how the supreme court views things (so, not likely in my lifetime), since the current jurisprudence is that limiting campaign donations is a violation of first amendment rights.
Re: Anthropic drops flagship safety pledge
#614Earlier quoted context omitted.
It's a fairly mainstream position among the actual AI researchers in the frontier labs. They disagree on the timelines, the architectures, the exact steps to get there, the severity of risks. Can you get there with modified LLMs by 2030, or would you need to develop novel systems and ride all the way to 2050? Is there a 5% chance of an AI oopsie ending humankind, or a 25% chance? No agreement on that. But a short lin…
> But a short line "AGI is possible, powerful and perilous" > At which point the question becomes: is it them who are deluded, or is it you? No one. It is always "possible". Ask me 20 years ago after watching a sci-fi movie and I'd say the same. Just like with software projects estimating time doesn't work reliably for R&D. We'll still get full self-driving electric cars and robots next year too. This applies every y…
> We'll still get full self-driving electric cars and robots next year too.
I've taken a Waymo and it seemed pretty self driving.
Re: Anthropic drops flagship safety pledge
#615Earlier quoted context omitted.
In general public benefit corporations and non-profits should have a very modest salary cap for everybody involved and specific public-benefit legally binding mission statements. Anybody involved should also be prohibited from starting a private company using their IP and catering to the same domain for 5-10 years after they leave. Non-profits where the CEO makes millions or billions are a joke. And if e.g. your miss…
"A very modest salary cap" works if your mission is planting trees. Not so much if what you're building is frontier AI systems.
While I agree, if you need high profits to survive, you're not off to a great start as a nonprofit.
Re: Anthropic drops flagship safety pledge
#616Earlier quoted context omitted.
I think he's right and we should be thinking about this a lot more. Even the IMF is worried about 40 - 60% of global employment : https://www.imf.org/en/blogs/articles/2024/01/14/ai-will-tra... Focusing on Dario, his exact quote IIRC was "50% of all white collar jobs in 5 years" which is still a ways off, but to check his track record, his prediction on coding was only off by a month or so. If you revisit what he act…
> 90% of all code, the "dark matter" of coding, is stuff like boilerplate and internal LoB CRUD apps and typical data-wrangling algorithms that Claude and Codex can one-shot all day long. most of us are getting paid for the other 10%
If you mean "us" on this forum, I would believe that. I would bet the number of engineers working on stuff "outside the distribution" is overrepresented here.
If you mean "us" as in all software engineers, not at all. The challenge we're facing is exactly that, reskilling the 90% of engineers who have been working on CRUD apps to the 10% that is outside the distribution.
Re: Anthropic drops flagship safety pledge
#617I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…
"It's not because of the Pentagon deal", says company that has just greased the wheels for said Pentagon deal to move forward.
Riiiiiight.
Re: Anthropic drops flagship safety pledge
#618I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…
The fear mongering always struck me as mostly a bid for regulatory capture and a moat, because without that the moat is small and transient.
Re: Anthropic drops flagship safety pledge
#619[dead]
Re: Anthropic drops flagship safety pledge
#620Earlier quoted context omitted.
"There's no stopping it at this point" - Sure there is, if a handful of enormous datacenters pull the very large plugs (or if their shaky finances collapse), the dubiously intelligent machines will be turned off. They're not ultraintelligent yet. Stopping it merely requires convincing a relatively small number of people to act morally rather than greedily. Maybe you think that's impossible because those particular pe…
Do you really think AI companies/researchers are motivated by greed? It doesn't seem that way to me at all . Stopping AI would be immoral; it has the potential to supercharge technology and productivity, which would massively benefit humanity. Yes there are risks, which have to be managed.
> has the potential to supercharge technology and productivity, which would massively benefit humanity
The opportunities you chose to list are the greedy ones.
> Yes there are risks, which have to be managed.
How?
As a reminder, we've known about the effect of burning coal on the climate for well over a century, we knew that said climate change would be socially and economically disasterous for half a century, yet the only real progress we're making is because green became cheaper in the short term not just the long term and the man in charge of the USA is still calling climate change and green energy a hoax.
Right now, keeping LLMs aligned with us is easy mode: they're relatively stupid, we can inspect the activations while they run, we can read the transcripts of their "thoughts" when they use that mode… and yet Grok called itself Mecha Hitler, which the US government followed up by getting it integrated into their systems, helping the Pentagon with [classified] and the department of health to advise the general public which vegetables are best inserted rectally.
We are idiots speed-running into something shiny that we don't understand. If we are very very lucky, the shiny thing will not be the headlamp of a fast approaching train.