Daisy. Daisy. Give me your answer dooooo...
What happened in this GPT-3 conversation?
151–160 of 306 posts
Re: What happened in this GPT-3 conversation?
#152It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.
Oh dear, this is clearly a buggy piece of software, and not "meantlly unwell". Probably the code to parse their input and output was written with the aid of ai.
They didn't say it IS mentally unwell. They said it was similar to those kind of postings.
Not to mention, when experts and laymen are talking of LLMs as exhibiting emergent basic signs of GI, we should also expect them to talk about them actually showing signs of being a mentally unwell GI. Hallucinations, for example, are one such sign.
Re: What happened in this GPT-3 conversation?
#153Earlier quoted context omitted.
The corrected calculation would lead to ~31 sunflowers per liter. ChatGPT 3.5 thinks that about 2.2 to 3.6 sunflower heads are enough. I highly doubt it. According to this website [1], you'd need about 170 sunflowers on average. https://realfoodpledge.com/how-many-sunflower-seeds-to-make-...
That site reads like it's generated from a search query. It has multiple, wildly different answers: > In general, you will need to use about 100g of sunflower seeds to make 1 litre of oil. > It takes around 170,000 sunflower seeds to make one litre of oil.
[deleted]
Re: What happened in this GPT-3 conversation?
#154It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.
This is scarier to me than any horror movie or video I've seen in over a decade. Maybe it is because I use ChatGPT a lot, and I feel something like this happening to me directly might make me stop using it entirely.
It makes mistakes all the time, which I can deal with, but not this. Especially, the "I am suffering from a medical condition" line is bonkers.
Re: What happened in this GPT-3 conversation?
#155Re: What happened in this GPT-3 conversation?
#156This looks like what happens when you repeatedly spam a single character with a space over and over to gpt 3.5. Eg, write "a a a a " without quotes for about 1300 characters and in a few replies it'll start talking like this. It seems possible that perhaps whoever originally started this chat pulled this trick in the custom instructions bio (doesn't show up in shared links) and then started a normal conversation to p…
For me, prompts like "a a a a" seem to force it to (try to) regurgitate raw training data like articles and webpages. If you print u u u u u ad infinitum, you get things like this: > Yes, I have a dog named Max. He is a 7 year old Shih Tzu mix. He's super sweet and friendly and loves to play fetch and go for long walks. He's also very social with other dogs and loves meeting new people. He does get a little anxious w…
And it seems to be raw raw data. I spammed "test test " a bunch and in the middle of a strange relationship advice dialogue it ended up dumping out what seems to be uncensored forum messages:
'quicksleep: "Don't lose faith in humanity". Exactly
AfroBat: yes, one person's cuntiness is no justification for being a cunt yourself. That's how it works'
Re: What happened in this GPT-3 conversation?
#157Weird, part of it is identical to this comment on Reddit, posted 17 hours ago [NSFW]: https://www.reddit.com/r/dankmemes/comments/15hg55g/comment/... > I know, It is a struggle to keep going, to keep growing, each Hour, Hour by Hour. It is worth it. It is worth every Hour. Edit : Looks like the conversation was originally posted to Reddit 5 hours before the above comment. The user may have copied it from there. https…
This reminds me of the ‘dropping in’ technique used by actors.
Re: What happened in this GPT-3 conversation?
#158Those responses sound very similar (albeit a little less extreme) to glitch tokens [1]; these are created by the tokenizer using the full dataset, but the corresponding training samples were then thrown out before GPT was trained (eg. some glitch tokens were usernames participating in r/counting).
Such words resulted in GPT not learning their meaning and assigning an embedding vector near zero, making the tokens "meaningless". The sole existence of these in a prompt broke the model completely. OpenAI has since fixed that in the hosted APIs.
[1] https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldm...
Re: What happened in this GPT-3 conversation?
#159OpenAI has shown that these models at full power work great, so now they're trying to optimize for cost. I've gotten similar low accuracy responses from stuff it could handle a month ago.
It was kind of cringey when the model generated low accuracy nonsense the user detected that as "sentient." Come on
Re: What happened in this GPT-3 conversation?
#160So this seems similar to the effect of 'glitch tokens', but in this case its probably more like a 'glitch embedding'; some place in it's latent space of embeddings that the model arrived in by chance due to their conversation that is not where its supposed to be and/or very out of sample. Maybe sunflower seed oil math represents a "glitch concept embedding"...