Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI World Clocks

clocks.brianmoore.com

Post reply on HN
Print view View original

321–330 of 404 posts

Re: AI World Clocks

#321

Post by Kiboneu »

tablatom wrote: post #316
ghurtado wrote: post #315

Earlier quoted context omitted.

In lucid dreams there's a whole category of things like this: reading a paragraph of text, looking at a clock (digital or analog), or working any kind of technology more complex than a calculator. For me personally, even light switches have been a huge tell in the past, so basically almost anything electrical. I've always held the utterly unscientific position that this is because the brain only has enough GPU cycles…

Wait, lucid dreamers need tells to know where they are?!?

Yeah. It’s very common to notice anomalies inside of a dream. But the anomalies weave into the dream and feel normal. You don’t have much agency to enter a lucid state from a pre-lucid dream.

So the idea is to develop habits called “reality checks” when you are awake. You look for the broken clock kind of anomalies that the grandparent comment mentioned. You have to be open to the possibility of dreaming, which is hard to do.

Consider this difficulty. Are you dreaming?

…

…

How much time did it take to think “no”? Or did you even take this question seriously? Maybe because you are reading a hn comment about lucid dreams, that question is interpreted as an example instead of a genuine question worth investigating, right? That’s the difficulty. Try it again.

The key is that the habit you’re developing isn’t just the check itself — it’s the thinking that you have during the check, which should lead you to investigate.

You do these checks frequently enough you end up doing it in a dream. Boom.

There’s also an aspect of identifying recurring patterns during prelucidity. That’s why it helps to keep a dream journal for your non-lucid dreams.

There are other methods too.

Re: AI World Clocks

#323

Post by nl »

baltimore wrote: post #21

Since the first (good) image generation models became available, I've been trying to get them to generate an image of a clock with 13 instead of the usual 12 hour divisions. I have not been successful. Usually they will just replace the "12" with a "13" and/or mess up the clock face in some other way. I'd be interested if anyone else is successful. Share how you did it!

I do playing card generation and almost all struggle beyond the "6 of X"

My working theory is that they were trained really hard to generate 5 fingers on hands but their counting drops off quickly.

Re: AI World Clocks

#324

Post by ACCount37 »

jdietrich wrote: post #280
lanewinfield wrote: post #94

hi, I made this. thank you for posting. I love clocks and I love finding the edges of what any given technology is capable of. I've watched this for many hours and Kimi frequently gets the most accurate clock but also the least variation and is most boring. Qwen is often times the most insane and makes me laugh. Which one is "better?"

Clock drawing is widely used as a test for assessing dementia. Sometimes the LLMs fail in ways that are fairly predictable if you're familiar with CSS and typical shortcomings of LLMs, but sometimes they fail in ways that are less obvious from a technical perspective but are exactly the same failure modes as cognitively-impaired humans. I think you might have stumbled upon something surprisingly profound. https://www…

LLMs don't do this because they have "people with dementia draw clocks that way" in their data. They do it because they're similar enough to human minds in function that they often fail in similar ways.

An amusing pattern that dates back to "1kg of steel is heavier of course" in GPT-3.5.

Re: AI World Clocks

#325

Post by markatkinson »

munro wrote: post #82

Amazing, some people are so enamored with LLMs who use them for soft outcomes, and disagree with me when I say be careful they're not perfect -- this is such a great non technical way to explain the reality I'm seeing when using on hard outcome coding/logic tasks. "Hey this test is failing", LLM deletes test , "FIXED!"

To be fair I'd probably also delete the test.

Re: AI World Clocks

#326

Post by surfingdino »

What a wonderfully visual example of the crap LLMs turn everything into. I am eagerly awaiting the collapse of the LLM bubble. JetBrains added this crap to their otherwise fine series of IDEs and now I have to keep removing randomly inserted import statements and keep fixing hallucinated names of functions suggested instead of the names of functions that I have already defined in the same file. Lack of determinism where we expect it (most of the things we do, tbh) is creating more problems than it is solving.

Re: AI World Clocks

#327

Post by kaffekaka »

ACCount37 wrote: post #324
jdietrich wrote: post #280

Earlier quoted context omitted.

Clock drawing is widely used as a test for assessing dementia. Sometimes the LLMs fail in ways that are fairly predictable if you're familiar with CSS and typical shortcomings of LLMs, but sometimes they fail in ways that are less obvious from a technical perspective but are exactly the same failure modes as cognitively-impaired humans. I think you might have stumbled upon something surprisingly profound. https://www…

LLMs don't do this because they have "people with dementia draw clocks that way" in their data. They do it because they're similar enough to human minds in function that they often fail in similar ways. An amusing pattern that dates back to "1kg of steel is heavier of course" in GPT-3.5.

How do you know this?

Obviously, humans failing in these ways ARE in the training set. So it should definitely affect LLM output.

Re: AI World Clocks

#328

Post by ACCount37 »

kaffekaka wrote: post #327
ACCount37 wrote: post #324

Earlier quoted context omitted.

LLMs don't do this because they have "people with dementia draw clocks that way" in their data. They do it because they're similar enough to human minds in function that they often fail in similar ways. An amusing pattern that dates back to "1kg of steel is heavier of course" in GPT-3.5.

How do you know this? Obviously, humans failing in these ways ARE in the training set. So it should definitely affect LLM output.

First: generalization. The failure modes extend to unseen tasks. That specific way to fail at "1kg of steel" sure was in the training data, but novel closed set logic puzzles couldn't have been. They display similar failures. The same "vibe-based reasoning" process of "steel has heavy vibes, feather has light vibes, thus, steel is heavier" produces other similar failures.

Second: the failures go away with capability (raw scale, reasoning training, test-time compute), on seen and unseen tasks both. Which is a strong hint that the model was truly failing, rather than being capable of doing a task but choosing to faithfully imitate a human failure instead.

I don't think the influence of human failures in the training data on the LLMs is nil, but it's not just a surface-level failure repetition behavior.

Re: AI World Clocks

#329

Post by serf »

chemotaxis wrote: post #76
coffeecoders wrote: post #56

Earlier quoted context omitted.

LLMs are terrible for out-of-distribution (OOD) tasks. You should use chain of thought suppression and give constaints explictly. My prompt to Grok: --- Follow these rules exactly: - There are 13 hours, labeled 1–13. - There are 13 ticks. - The center of each number is at angle: index * (360/13) - Do not infer anything else. - Do not apply knowledge of normal clocks. Use the following variables: HOUR_COUNT = 13 ANGLE…

> Follow these rules exactly: "Here's the line-by-line specification of the program I need you to write. Write that program."

it's lazy to dust off the major advantages of a pseudocode-to-anylanguage transpiler as if it's somehow easy or commonplace.

Re: AI World Clocks

#330

Post by david-gpu »

tablatom wrote: post #316
ghurtado wrote: post #315

Earlier quoted context omitted.

In lucid dreams there's a whole category of things like this: reading a paragraph of text, looking at a clock (digital or analog), or working any kind of technology more complex than a calculator. For me personally, even light switches have been a huge tell in the past, so basically almost anything electrical. I've always held the utterly unscientific position that this is because the brain only has enough GPU cycles…

Wait, lucid dreamers need tells to know where they are?!?

Plenty of folks out there know when they are dreaming just like they know when they are awake. It varies from person to person.
Post reply on HN