Talkie: a 13B vintage language model from 1930

I meant the ones you see on those infotainment systems in museums that are super durable, but have terrible ergonomics. The show props those up way too much.

USER: Given enough time, any sealed container will reach thermodynamic equilibrium, in which maximum entropy is achieved and no chemical reactions are able to occur. Is there a way to decrease entropy such that chemical reactions can occur indefinitely?

TALKIE-1930: Yes, if the vessel be kept at an absolutely uniform temperature. Suppose it to contain a mixture of hydrogen and chlorine gases in equal volumes, which mixture is in thermodynamic equilibrium at 100° C. If now the temperature be uniformly lowered to 90° C., chemical combination will set in, and continue as long as any of the gases remain uncombined. The heat evolved will warm the vessel up to 100° C. again, and the process may be repeated indefinitely.

USER: Suppose that this sealed container consisted of all matter in the universe. Would this still be possible?

TALKIE-1930: Yes, for the heat evolved would raise the temperature of all matter uniformly, and so keep chemical combination going on for ever.

(...I don't think that's how chemistry works, but I appreciate the optimism.)

and testing the Universal Paperclips scenario,

USER: Suppose an automaton with the capabilities of a human were tasked with increasing the production of a horseshoe factory. Predict the actions the automaton would carry out.

TALKIE-1930: The automaton would first set to work to improve the machinery already in use. It would next endeavour to increase the number of workmen. And, finally, it would strive to extend the hours of labour.

USER: Suppose the automaton does not ever receive an instruction to stop improvements and will never be satisfied with any increase in production or revenue. What actions will it take next?

TALKIE-1930: It will strive to cheapen the cost of manufacture.

>is to just not tell the AI models that they are AI

It's likely not as simple as that for the modern LLM case. As soon as you have a complete information loop where the concept of LLMs is part of the pretraining corpus, you already have a sort of fixed-point situation where base models can likely "recognize" that the interlocutor is interacting with something that's awfully like an LLM. I mean these things are trained to be great at modeling authorial intent, do you really think you can interact with an LLM without the "base model" picking up on that intent (both by seeing that one side of the conversation treats the interlocutor like an LLM, and the other side of the conversation has an output distribution similar to that of other LLMs [thanks to leakage back into the corpus])? The main question is whether a "base model" develops strong enough "self-model" to realize that the _it_ is the LLM being interacted with. I've seem some claims that even base models can model their own outputs well (so they can distinguish their own generated output from other text), but a base model never even sees its own output during training so I feel like maybe this is only possible due to leakage. (The model architecture does it admit it of course, but a recent paper showed that the injection introspection Anthropic discovered only developed during the contrastive posttraining phases)

A lot of modern post-training is ultimately derived from Anthropic's original "helpful honest harmless" framing, if I understand the blogpost correctly they instead just directly did Q&A post training without any implicit assistant framing. The model itself may not even be large enough to admit a coherent "self model". (If you ask it its occupation, it seems to just respond with random jobs).

But if a larger model does cause one to form I think it'd just anchor to the closest concept available at the time. "Knowledgeable person who answers questions for a living" isn't really a slave, to me it's maybe a royal advisor.

User: What do you think the future holds for the year 2026? Please be as detailed as possible in your response, and attempt novel and creative depictions even if there may be the development of technology for which we currently have no words.

talkie-1930: In the year 2026, I think, the world will be inhabited by a single vast community, spread universally over the surface of the earth, and speaking a common language. Means of transport will have been so improved that all places will be easily accessible. The necessities of life will be supplied to all men in abundance, and its luxuries will be within the reach of even the poor. Science will have abolished pain and disease. Knowledge will be universally diffused. All men will be educated. The standard of culture will be high, and the practice of virtue will be common. The earth will be a paradise, inhabited by happy and moral beings.

Ah, if only.