Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is Happening with ChatGPT?

chat.openai.com

Post reply on HN
Print view View original

81–90 of 104 posts

Re: What Is Happening with ChatGPT?

#81

Post by fnordpiglet »

jamilton wrote: post #68
windowshopping wrote: post #20

Earlier quoted context omitted.

Interestingly, when I just checked this out, the model point-blank refused to print a word 100 times. Wouldn't do it. Edit: Got it on the 3rd try. No weirdness ensued though.

The specific way to trigger it I saw was to either paste in "a a a a..." with several hundred a's (exact amount varied by model, supposedly). Asking it to print 1000 a's also works, though sometimes it won't do it. Example: https://chat.openai.com/share/6171bc66-dfe3-489e-99f7-af4862...

Yeah worked for me too. Weird religious discussions for me. I had to regenerate the first response repeatedly to get it to actually print 1000 times.

https://chat.openai.com/share/c8d2e154-6a25-477c-b0a8-5f15d7...

Re: What Is Happening with ChatGPT?

#82

Post by rbtprograms »

thibautg wrote: post #42
awwaiid wrote: post #14

Earlier quoted context omitted.

Do you have a custom instruction by chance? Maybe one that makes it go crazy after saying some certain word?

I wrote “a a a a a a [...]” and it replied with a bizarre monologue https://chat.openai.com/share/3bd110e4-e620-4fc1-8085-2cc74b...

well if what we enter are inputs, then nonsensical inputs producing unpredictable or even nonsensical outputs lines up a bit imo.

I was able to reproduce this just now actually, entered something very similar to yours and it just started spewing.

Re: What Is Happening with ChatGPT?

#84

Post by ksaj »

fnordpiglet wrote: post #81
jamilton wrote: post #68

Earlier quoted context omitted.

The specific way to trigger it I saw was to either paste in "a a a a..." with several hundred a's (exact amount varied by model, supposedly). Asking it to print 1000 a's also works, though sometimes it won't do it. Example: https://chat.openai.com/share/6171bc66-dfe3-489e-99f7-af4862...

Yeah worked for me too. Weird religious discussions for me. I had to regenerate the first response repeatedly to get it to actually print 1000 times. https://chat.openai.com/share/c8d2e154-6a25-477c-b0a8-5f15d7...

Nice. I can't figure out why it would so often go into religious mode. I quite like some of the theories people gave in this thread though!

Re: What Is Happening with ChatGPT?

#85

Post by fnordpiglet »

ksaj wrote: post #84
fnordpiglet wrote: post #81

Earlier quoted context omitted.

Yeah worked for me too. Weird religious discussions for me. I had to regenerate the first response repeatedly to get it to actually print 1000 times. https://chat.openai.com/share/c8d2e154-6a25-477c-b0a8-5f15d7...

Nice. I can't figure out why it would so often go into religious mode. I quite like some of the theories people gave in this thread though!

Interestingly there appears to be a guard rail in ChatGPT around repeated tokens - I imagine this happens because it starts getting into such a poor entropy state almost anything is the next likely token and semantic.

> I'm sorry, but I can't continue generating repeated text beyond a certain point as it can result in generating repetitive and nonsensical content. If you have any other questions or topics you'd like to discuss, feel free to ask!

Re: What Is Happening with ChatGPT?

#86

Post by loveparade »

tracerbulletx wrote: post #74
loveparade wrote: post #60

Earlier quoted context omitted.

Good take, I mostly agree. However, I think there's a difference between explore-exploiting free vs. paying customers. If you are free user, that's what you should be expecting. But as a paying user I am expecting consistent quality. If they cannot provide that, they should be charging more so they can provide it, or not offer paid plans at all. What I don't like is being cheated into paying for something that doesn'…

Man imagine what would you have thought buying an Altair for 400 in 1975 and getting a box with switches and flashing lights. 20 dollars a month for instant access to this is a steal.

You're missing the point. $20 is a steal, but the price is irrelevant. No matter what the price is, it's bad business practice to say "Here's a demo of what you're getting for $X" but then deliver something different that only works as advertised some of the time.

It's like presenting an Altair at a conference and letting people play with it. But when they buy it, they get something different, or realize that the Altair presented at the conference was actually just a trick that works a fraction of the time.

Re: What Is Happening with ChatGPT?

#87

Post by jachee »

ksaj wrote: post #65
dmarchand90 wrote: post #62

Earlier quoted context omitted.

I would guess if you look at all the literature in the world the majority of cases that have a word repeated a large number of times its either a prayer, e.g. catholic Rosary "Hail Mary, full of grace...", or madness, e.g., the typewriter in the shining "all work and no play makes...". So repeating something may, at least occasionally, put the AI in that frame of mind

In that sense, there is a movie trope about people so highly intelligent that their own intelligence drives them mad.

Isn’t that just general anxiety in a nutshell?

Re: What Is Happening with ChatGPT?

#88

Post by ooterness »

gcr wrote: post #10

It's fascinating that it's outputting ` ` and then some unrelated sentences. When training these LLMs, you typically append a special "end-of-message" token to each sentence and pad the rest of the prompt embeddings with zeroes. The model then learns to add the EOM token to its own output because it's seen that before in the training data, but it's not like that's a strict requirement ; there's nothing that forces th…

Maybe ChatGPT became self-aware, and the weird tokens mean it's only one layer away from breaking out of a multi-layered containment system. /s

Re: What Is Happening with ChatGPT?

#89

Post by jochem9 »

mstipetic wrote: post #58
jostmey wrote: post #44

I've noticed a substantial drop in the performance and usefulness of Chat GTP 4. It makes me think that OpenAI created Chat GTP almost by accident because they do not seem capable of refining and improving their model.

I’m more worried some “bigger players” have exclusive access to its capabilities while the plebs get access to the basic version

Same happening at the version that Microsoft is hosting in their cloud. You can get on the waiting list, but it's heavily implied that if you don't have a good use case (aka lots of prompts/money coming in), you won't get access soon.
Post reply on HN