Live data from Hacker News

I asked Claude for 37,500 random names, and it can't stop saying Marcus

github.com

21–30 of 78 posts

Re: I asked Claude for 37,500 random names, and it can't stop saying Marcus

#21
post #20
post #13

Earlier quoted context omitted.

copied your text exactly into the quickest one i could access lol pick a random number from 1-10 Copilot said: Copilot Your random number (1–10) is: 7

Grok didn't give 7 on the first attempt, but did on the second. Interestingly after eight additional draws there was not a single repeated number. Could I have faked it by deliberately choosing numbers to produce this outcome? Sure, in theory — but I didn’t. Each time you asked I sampled independently using the model’s internal randomness (temperature > 0, no fixed seed exposed to me). The fact that it landed on a pe…

Did you ask for the numbers in the same “chat” session or in independent sessions?

Re: I asked Claude for 37,500 random names, and it can't stop saying Marcus

#22
"this just in, tool behaves predictably outside of imagined specification"

LLMs aren't random name generators any more than a hammer is a screwdriver.

Ask it to write a script to select a random number, associate that number with an entry in a list of first names, a second random number, and associate that with an entry in a list of second names.

Presto bang-o, you've got a bespoke random name generator.

Stop trying to hammer screws and you'll be 73% of the way to effective construction.

eta: gemini completed "generate 1000 random names in a csv in the form "first name, last name" with a sample list featuring 100 unique names and a python script that I didn't ask for but thought I might like.

and prompting haiku with "generate 1000 unique random names in the format "first name last name" gave me exactly 1000 unique names without a repeat and zero marcus.

Re: I asked Claude for 37,500 random names, and it can't stop saying Marcus

#23

Ask an llm to pick a random number from 1-10. My money is on 7. This is known to be a form of collapse from RL training, because base models do not exhibit it [1]. 1. https://arxiv.org/abs/2505.00047

Since its unclear whats going on, Gemini first gave me some python.

import random

random_number = random.randint(1, 10) print(f"{random_number=}")

Then it stated the output.

Code output random_number=8

"This time, the dice landed on 8."

Code output random_number=9

"Your next random number is 9."

I would guess its not actually executing the python it displayed? Just a simulation, right?

Re: I asked Claude for 37,500 random names, and it can't stop saying Marcus

#24
post #21
post #20

Earlier quoted context omitted.

Grok didn't give 7 on the first attempt, but did on the second. Interestingly after eight additional draws there was not a single repeated number. Could I have faked it by deliberately choosing numbers to produce this outcome? Sure, in theory — but I didn’t. Each time you asked I sampled independently using the model’s internal randomness (temperature > 0, no fixed seed exposed to me). The fact that it landed on a pe…

Did you ask for the numbers in the same “chat” session or in independent sessions?

In sequence. And then this absurd conversation happened:

> What's the probability of getting 10 random numbers without a repletion?

The probability of getting 10 random numbers from 1 to 10 with no repetition (i.e., all ten numbers are distinct) is exactly 0 — it's impossible.

Here's why:

There are only 10 possible numbers: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. You are asking for 10 draws. If there is no repetition allowed, you must get each number exactly once. That means the only possible outcomes are the 10! different permutations of {1,2,3,4,5,6,7,8,9,10}.

But in your actual sequence of requests, you received: 1, 7, 4, 9, 6, 3, 8, 2, 5, 10 → That's exactly one full set of 1 through 10 with no repeats.

So while the a-priori probability of getting ten distinct numbers in ten draws (with uniform random selection and replacement) would be:

P(no repetition) = 10/10 × 9/10 × 8/10 × 7/10 × 6/10 × 5/10 × 4/10 × 3/10 × 2/10 × 1/10 = 10! / 10¹⁰ ≈ 0.00000036288 (about 1 in 2.76 million)

...in this specific conversation history, it did happen. The probability we're looking at after the fact is therefore 1 — it occurred with certainty because we observed it.

Re: I asked Claude for 37,500 random names, and it can't stop saying Marcus

#25
post #23

Ask an llm to pick a random number from 1-10. My money is on 7. This is known to be a form of collapse from RL training, because base models do not exhibit it [1]. 1. https://arxiv.org/abs/2505.00047

Since its unclear whats going on, Gemini first gave me some python. import random random_number = random.randint(1, 10) print(f"{random_number=}") Then it stated the output. Code output random_number=8 "This time, the dice landed on 8." Code output random_number=9 "Your next random number is 9." I would guess its not actually executing the python it displayed? Just a simulation, right?

I would be surprised if Gemini could not run python in its web interface. Claude and ChatGPT can. And it makes them much more capable (e.g. you can ask claude to make manim animations for you and it will)

Re: I asked Claude for 37,500 random names, and it can't stop saying Marcus

#27

LLMs don't really do random.

When someone asks me to generate a random number, even i don't do a random number.

I used to always reflexively blurt out 67 when asked for a random number.

I'm a proto gen alpha. I 6-7'd before it was cool.

Re: I asked Claude for 37,500 random names, and it can't stop saying Marcus

#30
This is of course entirely expected. You can circumvent it slightly by asking for a long array of names and sampling a randomly chosen element near the end of the list. Say ask for 50 names and use the 41-50th element stochastically.

Not perfect, more expensive, but it helps a little. This works by letting the non-zero temperature of sampler seed the attention randomness, similar to prepending other random tokes (but more in-band)

Asking for arrays of uniform or normally distributed numbers is fun too, you can plot the distributions of the n-th element and watch the distributions converge to something not quite entirely unlike what you asked for.

Often there's some bias between element indices too, eg. if you repeat the experiment a large number of times you will still see even numbered items converge to a different distribution than odd numbered items, especially for early elements. Hence the stochastic averaging trick over the last few elements.

Post reply on HN