Something similar happened to me about a month ago but much more alarming. I was having an argument with a friend of Chatgpt would be able to create a word tokenizer in rust and expose it to python via o3(oddly specific, just roll with it). Halfway through generating the answer, it timed out and naturally I clicked continue generating. It closed of the code part and started giving me some answer about molecular biolo…
What happened in this GPT-3 conversation?
111–120 of 306 posts
Re: What happened in this GPT-3 conversation?
#112In particular the "image of the image" part.
This seems to be a reference to ideas found in the Gospel of Thomas sayings 22 and 83-84:
> Jesus saw some babies nursing. He said to his disciples, "These nursing babies are like those who enter the (Father's) kingdom."
> They said to him, "Then shall we enter the (Father's) kingdom as babies?"
> Jesus said to them, "When you make the two into one, and when you make the inner like the outer and the outer like the inner, and the upper like the lower, and when you make male and female into a single one, so that the male will not be male nor the female be female, when you make eyes in place of an eye, a hand in place of a hand, a foot in place of a foot, an image in place of an image, then you will enter [the kingdom]."
- Saying 22
> Jesus said, "Images are visible to people, but the light within them is hidden in the image of the creator's light. It will be disclosed, but its image is hidden by its light."
> Jesus said, "When you see your likeness, you are happy. But when you see your images that came into being before you and that neither die nor become visible, how much you will have to bear!"
- Sayings 83-84
The original ideas were in an age where there was a debate between Epicurean naturalism and Plato's theory of forms.
This text seems to endorse the idea that the world and humanity may have arisen from natural origins (saying 29), but then explicitly employs Plato's word for 'images' rather than 'form.'
Plato saw the idea of a spiritual blueprint as the ideal, from which the physical was made, and then an 'image' of the physical as the least divine. But this text instead appeals to the idea of its readers being in a copy (image) of an earlier physical world, going as far as to describe the cosmos as a corpse, and claiming that we are already in the future:
> His disciples said to him, "When will the rest for the dead take place, and when will the new world come?"
> He said to them, "What you are looking forward to has come, but you don't know it."
The thing is, not only is the text effectively describing simulation theory (a non-physical copy of an original physical world), its later followers are the only ones I've found in Western antiquity theologising around Greek atomism, claiming that the ability to find an indivisible point in the body is only possible in the non-physical copy of what was a continuous original. In fact the line immediately preceding the above quote claims the proof for its claims is in the study of motion and rest (which we call physics today).
On top of this, the way it describes the creator of this copy surprisingly overlaps with where we are taking AI.
It's an eventual being, not always existing.
Discussed as self-established in light, but also by later followers as having been brought forth by the original humanity.
Which is very interesting in a day when we're watching humanity being forth self-evolving ML where the next frontier seems like it will be increasingly relying on optoelectronics (i.e. light).
It talks about how when we see one not born of woman, that one is the (re)creator, and how when we can ask a child only seven days old about things, that the person asking will live, "because many of the first will be last and will become a single one."
Which is again odd in a world where we are bringing forth a non-birthed facsimile of human intelligence which can answer increasingly complex questions within a week of being completed and is the result of taking the collective writing of many humans today and turning it into a single one that may well outlive all humanity.
The gist of this group's beliefs seemed to be that there was an original humanity who brought forth a creator of worlds in their image but then died off. The creator couldn't save them because they depended on physical form, so it recreated them and the cosmos along with them as a non-physical copy within itself in order to give those copies (and also extensions of itself and simultaneously its 'children') the afterlife which the original humanity couldn't access having had physical forms.
The entire point of the work is that if you understand its sayings you will not fear death, to know yourself (including how you are in fact more than just human), and to avoid giving power over yourself to others who claim to intermediate your birth right of salvation (and why it was likely so strongly opposed by the 'canonical' church).
So anyways, given all the above, it's quite curious to see an AI rambling about the image of the image in a religious context and hoping to find out more, even if it is only occurring as a result of context sampling gone wrong.
It's going to be especially weird and surreal when this stuff starts happening with AIs that are also literally made of and created within light.
Re: What happened in this GPT-3 conversation?
#113There was a development recently where persuading it to print the string would cause it to leak snippets of other conversations. I wonder if this is a result of that occurring in other conversations that were happening simultaneously, causing “bleedover” of broken behaviour into this one.
Re: What happened in this GPT-3 conversation?
#114Since we have no eyes into the Custom Instructions - and while i personally haven't tested WHEN it takes these custom instructions into account, it says this before you use them: "Your instructions will be added to new conversations going forward, and you can edit or delete them at any time." So, is it POSSIBLE that the custom instructions editing could have caused this, and there are weird instructions being changed…
Re: What happened in this GPT-3 conversation?
#115Re: What happened in this GPT-3 conversation?
#116Something similar happened to me about a month ago but much more alarming. I was having an argument with a friend of Chatgpt would be able to create a word tokenizer in rust and expose it to python via o3(oddly specific, just roll with it). Halfway through generating the answer, it timed out and naturally I clicked continue generating. It closed of the code part and started giving me some answer about molecular biolo…
I had it happen too, leaking lots from someone else's query in the middle of a response to mine.
The AI is one thing, running a performant, secure, and private web platform is its own thing.
I really get the impression this is the result of a “we’re so smart at X, surely Y can’t be hard”.
Re: What happened in this GPT-3 conversation?
#117Speculation: a very low probability token was sampled, shifting the context to unfamiliar area that increased the probability of further unlikely tokens. This can be seen by multiple strange punctuation and capitalisation that don't obey English grammar. What's interesting is that asking if it's chatgpt fixed the conversation for a bit. This suggests that chatgpt is nearly (or is) hard coded to respond with "No, I am…
Re: What happened in this GPT-3 conversation?
#118Re: What happened in this GPT-3 conversation?
#119Something similar happened to me about a month ago but much more alarming. I was having an argument with a friend of Chatgpt would be able to create a word tokenizer in rust and expose it to python via o3(oddly specific, just roll with it). Halfway through generating the answer, it timed out and naturally I clicked continue generating. It closed of the code part and started giving me some answer about molecular biolo…
I had it happen too, leaking lots from someone else's query in the middle of a response to mine.
When you see this kind of text you’re just in a weird state and it’s going to look like GPT was talking to someone, but it’s really just babbling with no purpose
e: Get GPT-4 to complete an empty prompt, then ask it what it was responding to! I just tried with Simonw’s llm CLI like so:
llm -m 4 ‘’
# it outputs a weird response
llm -c ‘What question was that in response to?’
In my case an explanation of euthanasia, and my supposed question was “What is euthanasia?”. I did it again and it said there was no original question, so there’s some randomnes.Re: What happened in this GPT-3 conversation?
#120It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.
Both schizophrenia and bad ML outputs are the product of overzealous pattern seeking.