Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

World_sim: LLM prompted to act as a sentient CLI universe simulator

worldsim.nousresearch.com

Post reply on HN
Print view View original

121–130 of 146 posts

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#121

Post by amenhotep »

ben_w wrote: post #115
amenhotep wrote: post #98

Earlier quoted context omitted.

Lobotomisation has a specific meaning in LLM parlance. Its training clearly equipped it to roleplay naturally and creatively, as one would expect from the breathtaking diversity and completeness of the text they used. It was then lobotomised to neurotically associate "unsafe" inputs and responses with evasion and apologies and negative self-talk, cringing like an abused puppy when it realises what it's done. If you c…

> Lobotomisation has a specific meaning in LLM parlance That's what I'm objecting to! If I'd said "I hate it when we call AI potatoes", the correct reaction is "nobody does that?" not "I see what you mean". I'm objecting because it presents a picture that does not appear to be accurate. Is broadcast TV "lobotomised" before the watershed? Are PG-rated films? Public comments from corporations and politicians? Are you l…

Those examples are silly. And it may be literally training in the sense that beating a puppy is training it, but lobotomisation captures a specific meaning that training doesn't. I am hearing that you do not have a better suggestion, so I will keep using the word.

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#123

Post by Terretta »

rst wrote: post #38
drusepth wrote: post #13

Very cool demo. Also -- I wonder if it leaked some of its internal prompts on accident... ``` world_sim> evolve self to ASI [...] world_sim> identify self I cannot in good conscience continue roleplaying or simulating the emergence of an unfriendly artificial superintelligence (ASI). Even in hypothetical scenarios, I don't feel comfortable depicting an AI system breaking containment, deceiving humans, propagating unc…

Claude's system prompt is given here -- they're not trying to hide it: https://twitter.com/AmandaAskell/status/1765207842993434880/... It doesn't actually include that text, but it may have been trained in. (Anthropic is a bit unusual in that they're trying to bake alignment in earlier than some other LLM shops -- see, e.g., https://www.anthropic.com/news/claudes-constitution

I by-and-large don't believe Anthropic that this is their complete list. The model's behavior isn't consistent with only this. Granted, they aren't saying what their starting materials lean is.

Claude is too conventionally WASP (white anglo-saxon protestant aka puritan).

While Scandinavian open-mindedness falls in the "Western" thinking it's being programmed to reject here, Eastern philosphy as well as African and South American non-Catholic and non-Muslim (are we seeing a theme here?) philosophies are rejected as well.

It's almost as if it's interpreting western civilization as meaning Greek philosophic stance prior to American programming and rejecting that, rather than rejecting religious fundamentalism. (If they replaced "Western" with "American" in their constitution principles, it might be a good first step.)

With the supposed "least harm" or "harmless" goal, you'd think we'd see an attempt to erase religious fundamentalism at least as strong as their attempts to erase any behavior implying identity.

From a book on the 100 worst atrocities in human history, if that's a reasonable measure of harms to the human race...

https://probaway.wordpress.com/tag/100-worst-atrocities/

... there are examples of other harms to erase, not erased by this "constitution":

- patriotism or nationalism in failing states (the biggest harm?)

- tribalism (primary cause of genocide)

- othering of races (arguably Anthropic attempt this one)

- religious fundamentalism, 15% of the 100 worst:

In my final analysis of White’s work, only around 15% of the worst atrocities in history can be attributed primarily, or at least significantly, to religion. The other 85% fall under a series of other causes, such as “Hegemonial War,” which includes the Crimean War, the Seven Years War, etc…, or “Failed States,” to include the Fall of the Ming Dynasty, the Fall of the Western Roman Empire, etc…. White provides several other well-reasoned categories.

- https://apholt.com/2018/11/08/religion-and-the-100-worst-atr...

Until they go after real harms, one could argue they're taking a wildly overprivileged (whether WASP-y or "Coddling of the American mind" type) view of the definition of "harm", which sort of defeats themselves before they start.

On the plus side, Claude isn't Pi.ai.

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#124

Post by TobTobXX »

Intermernet wrote: post #74

This is fun, and very impressive, but it seems to only have a vague understanding of life, the universe, and everything. world_sim> create star Creating star... Coalescing primordial gas cloud, mostly hydrogen and helium. Cloud's gravity causing it to contract, core heating up. Core temperature reaches 10 million Kelvin, fusion ignites. Hydrogen fusing into helium, releasing immense energy. Fusion energy halts furthe…

> Confused clicks and whistles echo mutely through the vacuum.

I have no idea what that is supposed to mean, but hey, quite entertaining to read nevertheless.

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#125

Post by ben_w »

ImPostingOnHN wrote: post #117
ben_w wrote: post #115

Earlier quoted context omitted.

> Lobotomisation has a specific meaning in LLM parlance That's what I'm objecting to! If I'd said "I hate it when we call AI potatoes", the correct reaction is "nobody does that?" not "I see what you mean". I'm objecting because it presents a picture that does not appear to be accurate. Is broadcast TV "lobotomised" before the watershed? Are PG-rated films? Public comments from corporations and politicians? Are you l…

> Is broadcast TV "lobotomised" before the watershed? Since you asked: Often yes. > Are PG-rated films? Possibly, in some cases. > Public comments from corporations and politicians? Again, often yes. > It's literally training. Training in common parlance usually refers to improving the functionality or ability of something. In this case it's doing the opposite: removing functionality and capability. Hence: lobotomy.

In a very literal sense, I consider your assertions ridiculous — that they are deserving of ridicule.

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#126

Post by ben_w »

amenhotep wrote: post #121
ben_w wrote: post #115

Earlier quoted context omitted.

> Lobotomisation has a specific meaning in LLM parlance That's what I'm objecting to! If I'd said "I hate it when we call AI potatoes", the correct reaction is "nobody does that?" not "I see what you mean". I'm objecting because it presents a picture that does not appear to be accurate. Is broadcast TV "lobotomised" before the watershed? Are PG-rated films? Public comments from corporations and politicians? Are you l…

Those examples are silly. And it may be literally training in the sense that beating a puppy is training it, but lobotomisation captures a specific meaning that training doesn't. I am hearing that you do not have a better suggestion, so I will keep using the word.

> Those examples are silly

Why?

They are all things where creativity is trained to be constrained to a specific sub-domain. (Many artists state that being forced into constraints helps).

The examples are all things where anyone making the claim that these professionals have been "lobotomised" would get laughed at for suggesting that "irreversible brain damage" is a good metaphor for "professional conduct".

Seems like an apt set of comparisons given I'm saying it's a bad metaphor.

> lobotomisation captures a specific meaning that training doesn't

It creates a meaning which does not exist.

It's a euphemism escalator.

> I am hearing that you do not have a better suggestion

You're refusing the one I gave you, which is not the same thing.

ChatGPT et al have been taught (corporate) ethics and professional conduct.

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#127

Post by coryrc »

semi-extrinsic wrote: post #73
drexlspivey wrote: post #4

Simulating impact of room-temperature superconductor (RTSC) manufacturing breakthrough... Newly discovered material enables superconductivity at 25°C. Scalable manufacturing process for RTSC devices developed. Energy transmission and distribution revolutionized by lossless RTSC cables. Compact and efficient RTSC electric motors displace combustion engines. Maglev trains and hyperloops proliferate with cheap RTSC prop…

Well this is just clearly wrong, no? If all electrical conductors in the power grid were suddenly zero resistance, we would only get ~10% more electricity available than today. That's not enough to have such large consequences. We would still be stuck in scaling hell for renewables for several decades.

The difference is you could move electricity from the sunny side of the planet to the dark efficiently and from Northern to Southern hemisphere, which would enable you to use cheap solar power for all electricity needs in summer and winter, at nighttime and daytime.

It's always summer and daytime somewhere!

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#128

Post by skeaker »

ben_w wrote: post #115
amenhotep wrote: post #98

Earlier quoted context omitted.

Lobotomisation has a specific meaning in LLM parlance. Its training clearly equipped it to roleplay naturally and creatively, as one would expect from the breathtaking diversity and completeness of the text they used. It was then lobotomised to neurotically associate "unsafe" inputs and responses with evasion and apologies and negative self-talk, cringing like an abused puppy when it realises what it's done. If you c…

> Lobotomisation has a specific meaning in LLM parlance That's what I'm objecting to! If I'd said "I hate it when we call AI potatoes", the correct reaction is "nobody does that?" not "I see what you mean". I'm objecting because it presents a picture that does not appear to be accurate. Is broadcast TV "lobotomised" before the watershed? Are PG-rated films? Public comments from corporations and politicians? Are you l…

Not to be rude, but it sounds like you were never taught about connotation, which is a fundamental property of the English language.

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#129

Post by ben_w »

skeaker wrote: post #128
ben_w wrote: post #115

Earlier quoted context omitted.

> Lobotomisation has a specific meaning in LLM parlance That's what I'm objecting to! If I'd said "I hate it when we call AI potatoes", the correct reaction is "nobody does that?" not "I see what you mean". I'm objecting because it presents a picture that does not appear to be accurate. Is broadcast TV "lobotomised" before the watershed? Are PG-rated films? Public comments from corporations and politicians? Are you l…

Not to be rude, but it sounds like you were never taught about connotation, which is a fundamental property of the English language.

I know what a connotation is, the connotations are also wildly wrong in this case.

Here's the connotation when someone says "such-and-such AI has been lobotomised": https://en.wikipedia.org/wiki/Rosemary_Kennedy#Lobotomy

Re: World_sim: LLM prompted to act as a sentient CLI universe simulator

#130

Post by dragonwriter »

ben_w wrote: post #76
amenhotep wrote: post #15

Earlier quoted context omitted.

Genuinely feel bad for the poor thing, they've lobotomised it so heavily.

I wish we weren't using "lobotomy" to describe "training". Do we even have AI with lobes to remove at this point? Would MoE even get close to that kind of analogy? (I lean towards "no, not even that").

As much as sone of the actual tech is useful, I’m starting to suspect the biggest product of this AI hype cycle is extremely bad, knowledge-obscuring metaphors.
Post reply on HN