Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happened in this GPT-3 conversation?

chat.openai.com

Post reply on HN
Print view View original

141–150 of 306 posts

Re: What happened in this GPT-3 conversation?

#141

Post by AgentME »

cma wrote: post #111
axegon_ wrote: post #99

Something similar happened to me about a month ago but much more alarming. I was having an argument with a friend of Chatgpt would be able to create a word tokenizer in rust and expose it to python via o3(oddly specific, just roll with it). Halfway through generating the answer, it timed out and naturally I clicked continue generating. It closed of the code part and started giving me some answer about molecular biolo…

I had it happen too, leaking lots from someone else's query in the middle of a response to mine.

It sees a lot of separate conversations in its training. It seems much simpler to assume that it's incorrectly learned that it should shift the conversation into a new one occasionally to match this tendency of the training data rather than a bug actually leaking and blending user conversations together.

Re: What happened in this GPT-3 conversation?

#142

Post by bondarchuk »

gumballindie wrote: post #133
breakingcups wrote: post #84

It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.

Oh dear, this is clearly a buggy piece of software, and not "meantlly unwell". Probably the code to parse their input and output was written with the aid of ai.

That is not what they said.

Re: What happened in this GPT-3 conversation?

#143

Post by jeroen »

smokel wrote: post #41
gus_massa wrote: post #3

> Can you reconsider this estimate it seems awfully high Is think the estimation is almost correct, but it has a bad calculation 920/0.02 = 46,000,000 (but the actual result is 46,000)

The corrected calculation would lead to ~31 sunflowers per liter. ChatGPT 3.5 thinks that about 2.2 to 3.6 sunflower heads are enough. I highly doubt it. According to this website [1], you'd need about 170 sunflowers on average. https://realfoodpledge.com/how-many-sunflower-seeds-to-make-...

That site reads like it's generated from a search query. It has multiple, wildly different answers:

> In general, you will need to use about 100g of sunflower seeds to make 1 litre of oil.

> It takes around 170,000 sunflower seeds to make one litre of oil.

Re: What happened in this GPT-3 conversation?

#145

Post by random3 »

This, along with other examples, are mostl likely "classic" (not ML-related) infrastructure bugs from low quality multi-tenancy implementation. I.e. whatever gets shared (servers or GPUs) doesn't get properly isolated. Sure enough it can be server-side sessions or actual GPU resources.

This surfaces the true level of safety/security and a good reflection of the reality compared to the projected "safety" OpenAI tries to persuade is needed.

Re: What happened in this GPT-3 conversation?

#148

Post by thesz »

Looks very much like infinite wisdom of neural Paul Graham.

http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Let me quote:

"...We can also play with the temperature of the Softmax during sampling. Decreasing the temperature from 1 to some lower number (e.g. 0.5) makes the RNN more confident, but also more conservative in its samples. Conversely, higher temperatures will give more diversity but at cost of more mistakes (e.g. spelling mistakes, etc). In particular, setting temperature very near zero will give the most likely thing that Paul Graham might say:

“is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same”

looks like we’ve reached an infinite loop about startups."

If I understand correctly, transformers are auto-correlating machines (their output gets fed to them as input) and it is not unexpected to see them get overly excited on something.

In any case, the same phenomena was observed a long time ago.

Re: What happened in this GPT-3 conversation?

#149

Post by coldtea »

joebiden2 wrote: post #121
hilsdev wrote: post #110

Earlier quoted context omitted.

Both schizophrenia and bad ML outputs are the product of overzealous pattern seeking.

Not sure why this was flagged. Unsubstantiated with citations, and not differentiated enough, sure, but not wrong. https://pubmed.ncbi.nlm.nih.gov/6725561/ This is just the first google result, there are countless others. And it is quite common knowledge.

Why would everything have to be "substantiated with citations"?

It's a comment in a discussion, not a research contribution. It should be judged (not "peer reviewed" as some research claim, but judged and accepted or not as a comment in a discussion) based on whatever it has merit to the best of the participants knowledge, like any other claim.

Not based on whether it comes with a bibliography. If someone wants, they can go and verify it.

Re: What happened in this GPT-3 conversation?

#150

Post by itishappy »

crooked-v wrote: post #92
FranchuFranchu wrote: post #87

Earlier quoted context omitted.

That's a really interesting idea. A LLM that tries to optimize engagement.

Really maximizing engagement would probably take at least allowing porn.

I believe Replika tried exactly that and it went about how you'd expect:

* They started optimizing for engagement, which meant making it extra horny for extra money (it sent pics).

* Horny bot forgets consent, refuses "no."

* Lawsuits, bans.

* Horny bot gets censored, main subreddit pins the suicide hotline for a while.

Replika ended up taking the app down and refunding everyone's money. Just kidding! It's still around and they're making a second app focused on "practicing flirting."

https://replika.com/

https://blush.ai/

Post reply on HN