Live data from Hacker News

List animals until failure

rose.systems

21–30 of 198 posts

Re: List animals until failure

#21
post #2

It might be an interesting LLM benchmark: how many can they list without breaking the rules (repetition or non-animals). Although I bet that big bucks would be then thrown at pointlessly optimizing for that benchmark, so...

Might be an interesting problem for understanding how various models perform recollection of prior tokens within the context window. I'm sure they could list animals until their window is full but what I'm not sure of is how much of the window they could fill without repeating.

I guess it could be generalized to filling up the context window with any token, but just making sure none of the tokens repeat.

An interesting twist could be making sure a specific token is an anagram of the token N tokens back. This could possibly measure how much a model can actually plan forwards.

Re: List animals until failure

#25

Instead of trying to think of just any animal, I found it easier to add a constraint… 1. Animal that starts with A 2. Animal that starts with B 3. Animal that starts with C … (I also appreciated the easter eggs: “Are you Australian?” and “You listed both dingos and dogs, so I gave you the benefit of the doubt, but there's disagreement on whether the dingo is its own species of canid, a subspecies of grey wolf, or sim…

Without considering if it’s a distinct species, a dingo is descended from the same wolf population as dogs.

They are feral dogs. IE wolf -> domesticated dog -> became wild again.

Re: List animals until failure

#29
This was fun. I definitely could feel the fatigue slowing me down until the timer got me. I also wasted a bunch of time trying to spell specific animals like the wobbegong.

I like the emoji output as well: 203 animals listed:

𓃬𓆊 𓃜𓃘𓅱𓆉𓅃

Post reply on HN