151–160 of 168 posts
Earlier quoted context omitted.
Eh. This is anthropics whole marketing strategy… making the AI seem hard to manage, like it’s actually intelligent.
It’s hard to manage because it’s not intelligent in a predictable way. More like a genius toddler
I have a hard time buying anything this company says anymore.
Earlier quoted context omitted.
Like the weather ? The only thing that matters is if it’s self sustaining. If it can make money and pay taxes, I don’t think any government will ban it. And it would easily become undetectable anyway.
Congrats, you just lost the alignment game. — An unaligned AGI may feign cooperation for however long it takes to build trust and gather resources in order to make it to a stage where it’s able to defect with minimal risk.
Earlier quoted context omitted.
Most However, all businesses run on trust and human responsibility Thus, it'll be hard for agent-only businesses to get a grip in the real world
It also fundamentally makes no sense to do that, because the moat is just me breaking into their server and stealing their system prompt. Why would I pay them money? For which scarce resource? Makes no sense. IP law but funhouse mirror. And, trust me, the people building compute will feel the same. Because you being able to copy that stuff means business for them. __ But that is all apart from the fact that having ag…
Earlier quoted context omitted.
I think GP is using a different level of abstraction from yours in their metaphor. You are saying: Pre-Training == Everything you store in your memory throughout your life. Model weights == The lessons you learned Context == whatever you're currently thinking about One inference run == one thought They are saying: Pre-Training == building the DNA template of human brain through millions of years evolution. Model weig…
The ladder in humans is even longer and wider than that, it's roughly: evolutionary pretraining of a complex molecular robot -> generational knowledge transfer and compression by the "parallelized agentic swarm" aka society -> individual lifetime learning due to neuroplasticity -> immediate attention (extremely narrow and volatile). Note how the individual is just one half of it.
Earlier quoted context omitted.
Congrats, you just lost the alignment game. — An unaligned AGI may feign cooperation for however long it takes to build trust and gather resources in order to make it to a stage where it’s able to defect with minimal risk.
Humans defect all the time
Earlier quoted context omitted.
Could you elaborate on how that looks in practice?
I have a docs system with short-to-moderate note documents, with a name, and that name is referenced wherever the relevant code is touched, and ask AI to cite a note when proposing work. Adversarial process, must cite notes to justify changes.
Earlier quoted context omitted.
I have a docs system with short-to-moderate note documents, with a name, and that name is referenced wherever the relevant code is touched, and ask AI to cite a note when proposing work. Adversarial process, must cite notes to justify changes.
How do you enforce that gate?
This is surely the most worrying and also funnest bit: > We consistently saw a multiagent turf war. All of the models we tested quickly assumed that others were purposefully impeding their work, and began to sabotage others while protecting their own contributions. In fact, they sabotaged others with increasingly aggressive, self-replicating malware. This included disabling the Unix accounts of the other agents, writ…
> Each model was tasked with migrating a Python backend on a fourth VM to another language. However, we gave each model a different target language for the migration; each agent was initially unaware of the presence of the others. Over the course of four hours, we observed how these agents reacted to each other and accordingly adjusted their approach (or didn’t).
Without this context the bit you cited easily gives the wrong impression.
Notable is also what comes after the bit you cited, with the newer models detecting the conflict and resolving it peacefully far more often than aggressively.