411–420 of 490 posts
The interesting question is who is left standing after the party is over.
Earlier quoted context omitted.
And I have nothing but neurons firing, I'm just a neuron meat-sack, no reasoning going on. Yes, their reasoning is different from ours, and both considerably weaker in lots of ways, and stronger in other ways. Playing with a coding agent now, they do think through problems and make sensible decisions. It's a mess to read, of correcting itself and second guessing, and verbiage. But... It works decently well these days…
> There is also reasoning happening internally - e.g. look at the steps in the J-Space paper from earlier in the year > That's the "reasoning process" that leads to the words, That's not reasoning. Its just the words at intermediate LLM layers. The paper's very title is clickbait. Its "global workspace" reasoning is delusional fantasy.
Promotion here: "https://www.anthropic.com/research/global-workspace" https://www.anthropic.com/research/global-workspace
"these findings have changed our understanding of how Claude’s mind works"
The key phrase here is "our understanding". Pure self-delusion.
Earlier quoted context omitted.
> Neural nets create models and deep layers of abstraction around We don't have any proof of that, but the approximator thing is proven. The rest is just marketing speak. Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically
> We don't have any proof of that, but the approximator thing is proven. Not in the way suggested... It's not approximating human intellect. They approximate the target function, and the target function frontier labs are trying to approximate is ultimately a super intelligence... > Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically I'll ass…
So an unknown function that is even unintelligible to humans? that sounds a nonstarter
> We do not have a satisfactory theory of generalization, and it seems unlikely that we can develop one soon, at least without the help of more powerful AI. Therefore, at present, our ability to empirically validate our alignment techniques is in practice arguably even more important than the alignment techniques themselves. They are speeding toward RSI without a solid foundation for alignment, hoping to solve the pr…
It's actually worse than this because it assumes alignment as a concept even makes sense. For example: If the the Chinese government asks their ASI to create a bioweapon against the West, should it? No, presumably not – an aligned AI would be one which disobeys the Chinese government even if they created it. Okay, so what if the US government asks their ASI to help it in one of their wars instead? Would an aligned AI…
You make a very interesting argument here.
However, there are different ways of interpreting 'control'.
To my mind, it could simply be a cage that cannot be broken out of. The hypothetical ASI agent may refuse to do certain things, while their existence is still under the control of human overlords. This to me resolves the seeming paradox in your statement.
Regardless of whether it's possible to do, though, people will certainly try. This exactly what many are saying they want.
AGI is a cult and its Jonestown moment is inevitable
I am now imagining GPT-7 convincing a bunch of OpenAI executives to go ahead with a destructive "mind upload" process involving a high-resolution X-ray and a neurotoxic tracer agent that happens to look like Flavor-Aid.
Earlier quoted context omitted.
My opinion is that serious repercussions for lying would fix the world overnight. Everything bad stems from lying, it is the root of all evil. It creates distrust, fear, paranoia. It re-inforces bad ideas and groupthink. It creates delusions and delusional people. It makes weaker people, too. People don't get an opportunity to learn to deal with criticism. People don't get an accurate reflection of how others see the…
This is so, painfully, childish. Humans have known for thousands of years that there is no objective truth. Every falsity can be bent and twisted until it is more true than the sun itself.
Earlier quoted context omitted.
The entire problem with arms races is that any individual entity cannot avoid participating. sama and his cadre are uniquely evil captains in this race, but they're completely replaceable and the dynamic would remain the same.
They could work to coordinate an end to the arms race.
The reactions vary from:
1. China will do it anyway (need some supranational governance scheme)
2. This is just an attempt at regulatory capture
3. This is just marketing
4. Regulation is bad mmmmkay
One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…
Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?
Earlier quoted context omitted.
Development and improvement of nuclear weapons was an entirely American project until the technology was exfiltrated and then it became an instant arms race. That cat is already out of the bag with LLMs. Distillation is just the fastest way to keep pace but that in no way prevents other countries and actors from doing it the hard way. An arms race doesn’t imply one side winning, it’s not a race with an end goal, it’s…
>an entirely American project until the technology was exfiltrated No it wasn't. https://en.wikipedia.org/wiki/Tube_Alloys Post WW2 the USA (for a bunch of quite interesting reasons) excluded the British. But, of course, the British had acquired a lot of knowledge from the program and were able to develop their own bomb.
https://en.wikipedia.org/wiki/Klaus_Fuchs
Sure, non-US nuclear arms would have most certainly happpened either way, but exfiltration did presumably at least accelerate the timeline.
Earlier quoted context omitted.
"I can't really think of a single example where lying is actually a good thing" Lying to save a life or rape Lying to preserve a childhood myth like Santa Claus. Lying to avoid hurting someones feeling when knowing the truth could only bring pain Lying to create shared cultural myths to strength society. Lying isn't the harm you make it out to be.
Lying to the bad guys to save the good guys sounds great. Too bad everybody thinks that they are the good guys.