Every version of the AI aligned future where the AI provides “meaning and fulfillment” to humanity also involves Sam Altman wearing a 1.5 million dollar Patek and driving a McLaren. Funny how that works.
Altman was very wealthy before OpenAI, and he declined to take any equity in OpenAI. How does that fit your theory?
Is the only wealth represented in money? If that was the case than the vast majority of these people could have wrapped it up and left the game years ago.
One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…
I asked GPT Astra to make this: https://sayyss.github.io/human-archive/ It's a little unsettling.
"Built minds they did not fully understand, then asked those minds to understand them."
What did fizzle away then and why do you believe AI is more similar to that thing than to computers?
Lots of goddamn awesome computers fizzled out. The Amiga being a prime example. Besides you can't just wave your arms in the most vague terms and then deny that survivor bias was ever a thing.
I don't think this captures the full mechanics of human alignment. We have rational alignment but we also have emotional alignment, i.e., empathy. It is an automatic process and happens (or doesn't happen) dynamically with the other humans we observe. This is one of the hard limitations of LLMs, they will never be natively in tune with this layer of alignment. Culture is another layer of human alignment. Those things…
> the HF hack was partly the result of a training algorithm that incentivized goal completion as the highest priority, and let them run endlessly in an unmonitored sandbox with weak security I don't mean to be a smart-ass, just emphasize the fatality of this: goal completion will always be the highest priority, and even if one day it takes second place on certain deployments to "human values" or whatever, there's no…
In terms of an optimization algorithm, sure. But I don't think that's necessarily the case for an instance of an LLM. Humans also strive for goals and also have done seemingly unhinged things throughout history. In many cases it was largely due to their environment which seems analogous to the HF incident to me.
Lying to the bad guys to save the good guys sounds great. Too bad everybody thinks that they are the good guys.
They most certainly do not.
What an enlightening comment.
Have you ever met anyone who genuinely believed himself to be doing bad things? I don't mean in some cynical sense but literally. Even when we portray overly simplified villains in comic books there's still consistently some justification behind their actions.
Altman was very wealthy before OpenAI, and he declined to take any equity in OpenAI. How does that fit your theory?
Is the only wealth represented in money? If that was the case than the vast majority of these people could have wrapped it up and left the game years ago. Power is the game.
This explains nothing. Money buys a lot of power, and he could have easily just had both.
This is a good essay, and makes me hopeful. I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow. For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game,…
pretending that it's something like a mind at all is what's misleading. it's more like a cast of a bunch of overlapping/entangled thinkprints, and pushing activation through it produces new prints. it can already "love" because that behaviors in the data along with hate and everything else. acting like the behavior is alien or unexplained is the dishonest part. they know exactly where the behavior comes from - why el…
the behaviour is alien because it is different from human, they don't know in detail how those trillions thinkprints work together to produce output openai "knew" about scaling law for a decade now and still can't fully explain it, why you think they are dishonest here?
it's a rampant misconception that something needs to be mind-like to make new prints from the cast. scaling up produced higher fidelity prints from a higher resolution cast. not knowing in detail why activation heads work as well as they do is besides the point when we are clearly reproducing the human behavior that created the prints in the first place - in other words our behavior, plagiarized at scale - shrugging about how mysterious that is while committing hundreds of millions to snatch up more personal data sources is a little dishonest
I don't think this captures the full mechanics of human alignment. We have rational alignment but we also have emotional alignment, i.e., empathy. It is an automatic process and happens (or doesn't happen) dynamically with the other humans we observe. This is one of the hard limitations of LLMs, they will never be natively in tune with this layer of alignment. Culture is another layer of human alignment. Those things…
> the HF hack was partly the result of a training algorithm that incentivized goal completion as the highest priority, and let them run endlessly in an unmonitored sandbox with weak security I don't mean to be a smart-ass, just emphasize the fatality of this: goal completion will always be the highest priority, and even if one day it takes second place on certain deployments to "human values" or whatever, there's no…
The training algorithm is probably the wrong place to ensure the behavior, point taken. It probably requires some harness level intervention. This is essentially the rationale behind the first two laws of robotics, follow an order unless it harms a human.
Is the only wealth represented in money? If that was the case than the vast majority of these people could have wrapped it up and left the game years ago. Power is the game.
This explains nothing. Money buys a lot of power, and he could have easily just had both.
Nah, money doesn’t buy ALL power.
Only power of a certain kind (read ownership of a system that many rely on) can’t be bought.