Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

old.reddit.com

Post reply on HN
Print view View original

51–60 of 68 posts

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#51

Post by armchairhacker »

People use this as evidence that ChatGPT is unlike human thinking, but we also have a randomness bias: https://youtu.be/d6iQrh2TK98?is=x6hiAqc0NJI7oeiE (referenced in one of the comments. tl;dr: when asked a number between 1-100, most pick a number with 7)

But ChatGPT’s bias is worse. It’s really not creative, and I think this hurts its output in “creative” cases, including stock photos and paid writing (ex: ML-assisted ads are even worse than unassisted ads), although not an issue in other cases like programming.

Now you may think - obviously that’s because the model has the same weights - but the problem is deeper and harder to solve. First, ChatGPT’s conversations are supposed to be “personalized”, presumably by putting users’ history and interests in the prompt; but multiple users reported the same fact about octopi. Maybe they turned off personalization, but if not, it’s a huge failure that ChatGPT won’t even give them a fact related to their interests (and OpenAI could add that specific scenario to the system prompt, but it’s not a general solution). Moreover, Claude, Gemini, and other LLMs also give random numbers between 7200-7500, while humans aren’t that predictable.

Since all LLMs are trained on the same data (most of the internet), it makes sense that all are similar. But it means that the commons are being filled with similar slop, because many people use ChatGPT for creative work. Even when the prompt is creative, the output still has a sameness which makes it dull and mediocre. I’m one of those who are tired of seeing AI-generated text, photos, websites, etc.; it’s not always a problem the first time (although it is if there’s no actual content, which is another LLM problem), but it's always a problem the 5th time, when I’ve seen 4 other instances of the same design, writing style, etc.

Some possible solutions:

- Figure out how to actually personalize models. People are different and creative, so the aggregate output of a personalized ML would be creative

- Convince most people to stop using AI for creative work (popular pressure may do this; even with people’s low standards I’ve heard Gen-Z tend to recognize AI-assisted media and rate it lower), and instead use it to program tools that enable humans to create more efficiently. e.g. use Claude Code to help develop an easier and more powerful Adobe Flash (that does not involve users invoking Claude Code, even to write boilerplate; because I suspect it either won’t work, or interfere with the output making it sloppier)

tl;dr: in case it isn’t already apparent, LLMs are very uncreative so they're making the commons duller. The linked example is a symptom of this larger problem

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#52

Post by helsinkiandrew »

throw310822 wrote: post #49
helsinkiandrew wrote: post #39

Earlier quoted context omitted.

LLMs aren't deterministic - they calculate a probability distribution of the potential next token and use sampling to pick the output.

Not really, the LLM is deterministic as far as I understand it, it's the sampling at the end that isn't. But the LLM can't prepare an even probability distribution to let the sampler decide randomly. It does reason deterministically and commits to a certain output.

> the LLM is deterministic as far as I understand it, it's the sampling at the end that isn't.

I guess it depends how you define the LLM: you could say it was the model/NN and the sampler is an extra added on, but a lot of people would name the model+sampler+system prompt+RLHF tuning (which would include the sampler) as the LLM.

The OP was talking about ChatGPT generating fixed output, not an internal model

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#53

Post by raphman »

pcblues wrote: post #43
raphman wrote: post #30

Ask ChatGPT or any other LLMs to give you ten random numbers between 0 an 9, and it will give you each number once (most of the time). At most, one of the digits may appear twice in my experience. Actually, when I just verified it, I got these: Prompt: "Give me ten random numbers between 0 and 9." > 3, 7, 1, 9, 0, 4, 6, 2, 8, 5 (ChatGPT, 5.3 Instant) > 3, 7, 1, 8, 4, 0, 6, 2, 9, 5 (Claude - Opus 4.6, Extended Thinkin…

"These look really random" - I hope I missed your sarcasm. That is so far from random. Think of tossing a coin and getting ten heads in a row. The probability of not repeating numbers in 10 numbers out of 10 is huge, and not random. Randomness is why there is about a 50% chance of 2 people in a class of about thirty having a birthday on the same day. Apple had to nerf their random play in iPod because songs repeated…

Oh yes, /s.

(I thought this was obvious and absolutely agree with your explanation.)

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#54

Post by armchairhacker »

pcblues wrote: post #37

This is what I hate about people trusting it. If you rely on AI to operate in a domain you don't man-handle, you will be tricked, and hackers will take advantage. "AI! Write me gambling software with true randomness, but a 20% return on average over 1000 games" Who will this hurt? The players, the hackers or the company. When you write gambling software, you must know the house wins, and it is unhackable.

This example isn't good, because (while I'm sure there would be security holes) ChatGPT writes a random number program fine.

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#56

Post by gzread »

throw310822 wrote: post #26
gzread wrote: post #19

Earlier quoted context omitted.

By emitting a next token distribution with a 10% chance of 0, 10% chance of 1, etc. Also it's an LLM, not a brain.

Interesting. So you expect it to "not think" and simply produce a value corresponding to "it's the same to me", knowing that it will be translated into an actual random value. Instead, exactly as a person would do, it does think of a specific number that feels random in that particular moment.

If "a random number is" is followed by all digits equally in the training dataset, then emitting a uniform distribution should minimise cross-entropy, right?

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#57

Post by aaron695 »

pcblues wrote: post #37

This is what I hate about people trusting it. If you rely on AI to operate in a domain you don't man-handle, you will be tricked, and hackers will take advantage. "AI! Write me gambling software with true randomness, but a 20% return on average over 1000 games" Who will this hurt? The players, the hackers or the company. When you write gambling software, you must know the house wins, and it is unhackable.

[dead]

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#58

Post by throw310822 »

gzread wrote: post #56
throw310822 wrote: post #26

Earlier quoted context omitted.

Interesting. So you expect it to "not think" and simply produce a value corresponding to "it's the same to me", knowing that it will be translated into an actual random value. Instead, exactly as a person would do, it does think of a specific number that feels random in that particular moment.

If "a random number is" is followed by all digits equally in the training dataset, then emitting a uniform distribution should minimise cross-entropy, right?

Er, should it? Even of you trained an LLM exactly over this type of question, if the sequence to predict in the training data is really random, then any output is equally wrong. Even if the output is a fixed "1234" or "0000". There is no signal to train on, not even one that favours an equal distribution.

On the other hand, LLMs show they know very well what a random number is and the fact it just shouldn't look like anything in particular, so they strive to come up with a number that doesn't look like anything in particular. Which happens to be always the same number given the starting conditions.

Re: Ask ChatGPT to pick a number from 1-10000, it generally selects from 7200-7500

#60

Post by HeavyStorm »

pcblues wrote: post #37

This is what I hate about people trusting it. If you rely on AI to operate in a domain you don't man-handle, you will be tricked, and hackers will take advantage. "AI! Write me gambling software with true randomness, but a 20% return on average over 1000 games" Who will this hurt? The players, the hackers or the company. When you write gambling software, you must know the house wins, and it is unhackable.

You just went and created the worst example. The model knows how to create an rng, that's not it weakness. In fact, if you give it a random mcp it won't do that.
Post reply on HN