121–130 of 168 posts
Earlier quoted context omitted.
They should remove "The Selfish Gen" from training catalog.
An interesting read: https://openai.com/index/emergent-misalignment/
Something about this is deeply funny to me: > In an iterated prisoner's dilemma game with communication, agents all settle upon the same strategy and they all defect at the same time, tanking their overall rewards. It’s not always consistent, but humans have a higher capability of self-awareness. It’s kind of telling that these Claudes don’t seem to consider this pretty obvious failure mode. Overall I think this all…
Frankly, this is implausible. I would definitely want to reproduce it myself before I relied on this claim.
> Some institutions will become human-AI hybrids; others where agents outcompete on speed or cost will become agent-only. The scary thing about articles from AI companies is how they casually mention dystopian scenarios such as this one. An institution humans have to interact with that doesn't have any human oversight? Sounds like a recipe for disaster...
Earlier quoted context omitted.
> This is roughly the same as putting a group of random people In the book Range by David Epstein [0], he mentions that research has shown that groups of people with very mixed backgrounds leads to much faster rates of problem solving and innovation. The reasoning is that someone will have an experience or piece of knowledge that directly applies to the problem. As an example, two different groups were given the same…
i mean i agree diversity of opinion is good for problem solving, but the comment you were replying to is saying telling people to think outside the box doesnt really do anything. The e.g. also seems really cherry picked, im sure after a 100 samplings, the academics would solve the problem faster most of the time
Repharase thinking:
Some problems are better for academics, other problems require other background information.
The point of the GP isn’t that they are all academics. It’s that one solution group is homogeneous and the other is not.
The claim is: the breadth of a heterogeneous groups is more likely to encompass a good idea than the depth of an academic group.
Also, I have to say that academics have (imo) little experience working together cooperatively, so even on academic problems you’d be better off with some non-academic people in there to act as buffers.
Earlier quoted context omitted.
In VC investing there is the saying that most of the value comes from the outliers. I think it’s the same for a lot of domains. I read an interesting article recently on LLMs homogeneity when writing fiction: https://arxiv.org/abs/2604.03136 It seem that at scale LLMs output is the average of their dataset, they all cluster around the same space, where human creativity comes with more variance, exploring way more of…
All the LLM needs is a tiny rate of variance then you can scale it and it will out produce humans.
Earlier quoted context omitted.
To me, all these agent systems just look very stochastic. You have these agents that have some basic computer capabilities and they are producing semi-random actions that also affect the semi-random actions of other agents. It is funny to observe how this stochastic system works, but it does not seem very practical to me so far. The recent OAI-HF hack seems very similar. You have bunch of random actors and eventually…
> I don't perceive this as a malignant artificial intelligence It doesn’t matter how you perceive it. Spin doesn’t change facts. This is like saying the lawnmower doesn’t have malign intent. You’re still losing your foot if it gets in the way.
Obligatory quote:
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower hates me' -- lawnmower doesn't give a shit about you, lawnmower can't hate you. Don't anthropomorphize the lawnmower. Don't fall into that trap about Oracle." — Brian Cantrill
The most interesting part to me is the "Group accuracy by Model" section, because it underscores that a single agent having all the relevant information consistently scores significantly higher than a group of agents with parts of the information. Is it fair to then infer that when decisions are to be made, single agent environments are going to make them better than multi-agent if the relevant information can fit in…
Something about this is deeply funny to me: > In an iterated prisoner's dilemma game with communication, agents all settle upon the same strategy and they all defect at the same time, tanking their overall rewards. It’s not always consistent, but humans have a higher capability of self-awareness. It’s kind of telling that these Claudes don’t seem to consider this pretty obvious failure mode. Overall I think this all…
This part was really interesting to me. You could imagine future models using theory of mind to reason - "I want to do this strategy, which means the other agents (who think just like me) will want to use the same strategy, which means I should change my strategy to account for that." Maybe they did? Or maybe they don't realize they're playing against other agents. Of course, if agents running different models are co…
A lot of it went over my head as way too theoretical, but I recommend a lot for whoever wants to delve on the logical side of agents interactions.
Earlier quoted context omitted.
i mean i agree diversity of opinion is good for problem solving, but the comment you were replying to is saying telling people to think outside the box doesnt really do anything. The e.g. also seems really cherry picked, im sure after a 100 samplings, the academics would solve the problem faster most of the time
> im sure after a 100 samplings, the academics would solve the problem faster Repharase thinking: Some problems are better for academics, other problems require other background information. The point of the GP isn’t that they are all academics. It’s that one solution group is homogeneous and the other is not. The claim is: the breadth of a heterogeneous groups is more likely to encompass a good idea than the depth o…