That would not be intuition that is just randomness. Intuition is not randomness. A jump in intuition comes from automatic processes reorganising the relational structure of conceptual models. There is no reorganisation of the model durring inference.
> There is no reorganisation of the model durring inference. one could argue that model can reorganize / interact with prior knowledge captured in text form (edit files) hence there can be reorganisation
Re-deriving the start of the art cleanly with fewer postulates is good!
And so I'm skeptical of TFA's claim that "creativity _isn't [just]_ compression" Imho there's also a clean argument against the existence of LLM-understanding: chatbots have been unable to summarize to experts (see my reply to you in the other thread) their own findings. Even after prolonged interrogation. They were unable to _compress_ their own findings. Thus they might not actually understand what they have actual…
I'm not sure, many chat bots summarise just fine as far as I can tell, on subjects I know a lot about.
Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
bro, it is a token predictor. You had me in the first half, language is humanity's greatest invention and all of the LLM "wins" can be attributed to existing language(imo). But no it's not going to have a vision like Einstein because it doesn't have a brain it is a token predictor.
Or, ask yourself the following question: If you read every single book there is about The Grand Canyon, and watched every single video and/or documentary about The Grand Canyon, do you believe that you have fully experienced The Grand Canyon? Or do you just have to be there to fully experience it. I dunno. Substitute in whatever you want for "The Grand Canyon". Maybe climbing Mount Everest or walking on the Moon. The…
I think I could get very close via video. But that depends on a lot of my non-canyon experience moving around the world, and LLMs are very flawed in their ability to input video, so I think they would not get nearly as close.
I've been to a few natural wonders (including the Grand Canyon) that I saw in advance on video. At least for me it isn't close at all. Even if audiovisual elements could be near-perfectly reproduced by video (imo not even close with modern tech, no screen is capturing the brilliance of sunlight), you aren't capturing the temperature, the feel of wind or rain, the smell of the plants around you, etc.
If you think LLMs can't "jump," watch Terence Tao engage with ChatGPT to try to understand the intellectual leaps achieved by Claude Fable 5 in finding a counterexample to the Jacobian conjecture https://chatgpt.com/share/6a5fdc7a-d6f8-83e8-bbea-8deb42cfed... .
After that you may revise your opinion.
Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
Or, ask yourself the following question: If you read every single book there is about The Grand Canyon, and watched every single video and/or documentary about The Grand Canyon, do you believe that you have fully experienced The Grand Canyon? Or do you just have to be there to fully experience it. I dunno. Substitute in whatever you want for "The Grand Canyon". Maybe climbing Mount Everest or walking on the Moon. The…
No way. You get to the rim and only then you realize how huge it is. And then you still have to go down into the canyon.
Einstein seems to be so conveniently dismissive that he knew about the seminal Michelson-Morley experiments but he probably knew about it too well [1]. Einstein is not the first great scientist who are in denial of other important prior contributions, and he also not the last one. Newton also probably knew too well about Al-Haytham (Alhazen), arguably the father of modern science, and his breakthrough experiments but…
Was the ethical requirement to discuss and acknowledge predecessors work as developed in Newton's time as in our own? I would naturally expect not because it would seem to me to be the kind of thing that develops over time, but I could be wrong as I am not a historian of science.
It reminds me a bit of an anthropology question I've heard brought up a lot that's sort of the dark inverse of xkcd 1053 [0], that many things that "everybody knows" tend to get lost over enough time. You don't need to properly cite experiments or lectures or books that "everybody knows". Especially when the work you do essentially replaces those things in the canon, such connections sometimes get entirely lost. To an anthropologist working in some ancient language the variations of the phrase "as everybody knows" are often a missing or lost citation today. The places where it is so implied that you have the same education or background or textbook as the author to not even warrant an "as everybody knows" is an ever worse thing because you don't even have the marker that you are missing a citation.
Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
bro, it is a token predictor. You had me in the first half, language is humanity's greatest invention and all of the LLM "wins" can be attributed to existing language(imo). But no it's not going to have a vision like Einstein because it doesn't have a brain it is a token predictor.
>> Specifically, I focused on the equivalence principle—a key axiom that Einstein formulated through thought experiments grounded in his physical intuition. It's weird because the equivalence principle is very unintuitive. Aristotle's Mechanics does not have it. It took almost two thousand years to discover inertia that is the most simple version of the equivalence principle. Einstein understood the idea of the the e…
> If we continue with Einstein, the photoelectric effect does not aperar in real life. Of course it does. How do you think your phone camera works?
But he didn't have one :) I guess unless you use the raw images, I think the cameras fix a lot of the noise caused by the Poisson distribution. I imagine a movie where Einstein get bored in the patent office and decide to open a photography shop, perhaps with very grainy underexposed photography is possible to notice the quantization of photons, and then he gets a Nobel prize.
Another bad idea for a movie is a watermillpunk universe, where during a practice on a hot day the best ever curling player discover inercia.
I think I could get very close via video. But that depends on a lot of my non-canyon experience moving around the world, and LLMs are very flawed in their ability to input video, so I think they would not get nearly as close.
I've been to a few natural wonders (including the Grand Canyon) that I saw in advance on video. At least for me it isn't close at all. Even if audiovisual elements could be near-perfectly reproduced by video (imo not even close with modern tech, no screen is capturing the brilliance of sunlight), you aren't capturing the temperature, the feel of wind or rain, the smell of the plants around you, etc.
But you already know those feelings from being outside elsewhere. You get a unique combination there, but that's only worth so much.