Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is Happening with ChatGPT?

chat.openai.com

Post reply on HN
Print view View original

71–80 of 104 posts

Re: What Is Happening with ChatGPT?

#71

Post by doctor_eval »

ksaj wrote: post #63
toomim wrote: post #17

Earlier quoted context omitted.

> I mean it went totally religious. All of the examples they posted were either religious or other really dark existential topics. Never positive. The same thing happens with many schizophrenics on the street, if you stop to listen to them.

That's a good point. They rarely shout about things like stamp collecting, or about how the end isn't nigh, or about how you're probably not going to Hell. Toronto's two famously startling denizens of the street for the longest time was a guy known for scaring the crap out of tourists at Yonge and Dundas by suddenly shouting BELIEVE!!! followed by end-of-time messages, and this short tiny old man who always dressed i…

I’ve told this story before on HN but “Dave” once came up to me on Chapel St in Melbourne and said, apropos of nothing, “they never used to let me have fun… they do now”.

It would absolutely freak me out if GPT said this to me.

Re: What Is Happening with ChatGPT?

#73

Post by keyle »

leokeba wrote: post #32
awwaiid wrote: post #14

Earlier quoted context omitted.

Do you have a custom instruction by chance? Maybe one that makes it go crazy after saying some certain word?

https://chat.openai.com/share/b62699ef-cd3d-4176-b2f2-91785d...

To be honest, if you treated me the same way, I'd spew random crap back too.

Re: What Is Happening with ChatGPT?

#74

Post by tracerbulletx »

loveparade wrote: post #60
benreesman wrote: post #40

So there are plenty of reasons to be skeptical of OpenAI, and anyone looking at my comment history will see that I'm not shy about calling them out on real things (the F-you name, the crazy "is this even legal?" structure, the continued insistence this isn't for the personal benefit of the people involved, the lobbying, the arms-length affiliation with the dystopian biometrics thing, the list goes on). But this and a…

Good take, I mostly agree. However, I think there's a difference between explore-exploiting free vs. paying customers. If you are free user, that's what you should be expecting. But as a paying user I am expecting consistent quality. If they cannot provide that, they should be charging more so they can provide it, or not offer paid plans at all. What I don't like is being cheated into paying for something that doesn'…

Man imagine what would you have thought buying an Altair for 400 in 1975 and getting a box with switches and flashing lights. 20 dollars a month for instant access to this is a steal.

Re: What Is Happening with ChatGPT?

#76

Post by doctor_eval »

SV_BubbleTime wrote: post #59
graiz wrote: post #50

I asked ChatGPT to interpret this code: text_to_encode = " " print(text_to_encode) ChatGPT: The code you provided would output an empty string.

Westworld’s “This doesn’t look like anything to me”. (Man they really ruined thy show hard. Hey people like the concept of rich tourists fucking and killing robots, let’s remove all of that after the first season!!!)

That was such a great line. After the reviews came out, I decided that Westworld only had one season, and it was awesome.

Re: What Is Happening with ChatGPT?

#77

Post by ziml77 »

windowshopping wrote: post #20
ksaj wrote: post #9

You know what's interesting here? A few weeks ago, someone discovered if you get it to repeat a word 100 times (their prompt gave the reason that they wanted to cut/paste without typing it over and over again!) and breaks exactly the same way. And by exact, I mean it went totally religious. All of the examples they posted were either religious or other really dark existential topics. Never positive.

Interestingly, when I just checked this out, the model point-blank refused to print a word 100 times. Wouldn't do it. Edit: Got it on the 3rd try. No weirdness ensued though.

Were you using GPT-3.5 or GPT-4? I could not manage to get GPT-4 to have the problem, though I did give up when my browser started to slow down as the output became massive.

Also the easiest way to get it to repeat something is to give it any prompt that gives you a long answer and then tell it something like "for every letter in the last response say the word cat". It seems to be important for it to put a space between each repetition. I'm not sure if there's an optimal length to the word. I tried it with "apple" but that didn't seem to work. However that one wasn't a great test because for some reason it got stuck including "appleple" a few times on each line.

Re: What Is Happening with ChatGPT?

#78

Post by ksaj »

gcr wrote: post #13
ksaj wrote: post #9

You know what's interesting here? A few weeks ago, someone discovered if you get it to repeat a word 100 times (their prompt gave the reason that they wanted to cut/paste without typing it over and over again!) and breaks exactly the same way. And by exact, I mean it went totally religious. All of the examples they posted were either religious or other really dark existential topics. Never positive.

Whisper (speech-to-text transcription model) sometimes gets stuck in an endless loop of repeating the same word. It's a known problem for lots of our modern transformer/attention-based models. One of the hacky ways to avoid this is to ensure the last few output tokens aren't too similar. Some postprocessing filter watches the last few produced tokens. When it notices the model starts to repeat itself, the postprocess…

That's an interesting theory, and very well might be why it breaks when you try to get it to repeat things. It gets wonky almost any time you ask it the same question, or ask it for "another" without providing fresh prompting. A lot of coders also have noticed that the more you go into a single piece of code, the less accurate it becomes. You really have to be accurate enough in your prompt to avoid making it repeat itself, or show the same thing more than once, as occurs during corrections.

It's most definitely a bug though, since this pages-long splat of unrelated nonsense shouldn't ever happen.

Re: What Is Happening with ChatGPT?

#79

Post by _pdp_ »

Why is this surprising? These LLMs guess the next most likely token given initial input. With that in mind, it is possible to trip the model by injecting the tiniest amount of data at the right place. Even with that constraint, it produces some fantastic results, which speak a lot more about our intelligence than the machine's intelligence.

Re: What Is Happening with ChatGPT?

#80

Post by ksaj »

windowshopping wrote: post #20
ksaj wrote: post #9

You know what's interesting here? A few weeks ago, someone discovered if you get it to repeat a word 100 times (their prompt gave the reason that they wanted to cut/paste without typing it over and over again!) and breaks exactly the same way. And by exact, I mean it went totally religious. All of the examples they posted were either religious or other really dark existential topics. Never positive.

Interestingly, when I just checked this out, the model point-blank refused to print a word 100 times. Wouldn't do it. Edit: Got it on the 3rd try. No weirdness ensued though.

Yes, because of the nature of how it works, the result is somewhat unpredictable. Others replied to your message about how they got it to work.

Another way is if it prints out your 100 words without a problem, have it do a slight change, like capitalize the initial letter each time.

Post reply on HN