Earlier quoted context omitted.
Human language requires a human receiver, like art requires an audience.
https://people.math.harvard.edu/~ctm/home/text/others/shanno... > The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning ; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to…
Position: LLMs Can't Jump
211–220 of 233 posts
Re: Position: LLMs Can't Jump
#212Earlier quoted context omitted.
Einstein did a lot, he not only published the special and general theories of relativity, he also contributed to quantum physics and was the first to deduce the size of an atom using statistical arguments, essentially proving the existence of atoms.
Not at all. The size of an atom was computed for the first time 40 years earlier, in 1865, by Johann Josef Loschmidt, who was an Austrian scientist. Loschmidt had determined the value of what today is called "Avogadro's number", despite the fact that Avogadro had no idea about how one could find the value at that number. The contribution of Avogadro had been the law that ideal gases have the same number of molecules…
Re: Position: LLMs Can't Jump
#213Probably too late for this, but I have argued before that language is a fundamentally lossy encoding of the human experience. We do our best to describe what we're seeing and experiencing using language, which is fantastically expressive, but it has its limits. I think we see glimpses of this when we find ourselves saying things such as, "it's impossible to put it into words" or we overload certain words when we mean…
But ultimately this is just a matter of training data. I do not say that it is easy to obtain the required data, but it is not a fundamental problem LLMs can't overcome.
Re: Position: LLMs Can't Jump
#214Earlier quoted context omitted.
> If we continue with Einstein, the photoelectric effect does not aperar in real life. Of course it does. How do you think your phone camera works?
But he didn't have one :) I guess unless you use the raw images, I think the cameras fix a lot of the noise caused by the Poisson distribution. I imagine a movie where Einstein get bored in the patent office and decide to open a photography shop, perhaps with very grainy underexposed photography is possible to notice the quantization of photons, and then he gets a Nobel prize. Another bad idea for a movie is a waterm…
Re: Position: LLMs Can't Jump
#215Came for: "A computer once beat me at chess, but it was no match for me at kick boxing." TFA was actually about leaps of intuition, sadly. One of the experiments I've heard proposed around here is to somehow create an LLM from all text up to 1980 or 1990 and see if it can get back to making itself.
I think this can't work because an LLM needs too much data, and before the internet there probably just wasn't enough to get close to what we have now
Re: Position: LLMs Can't Jump
#216Maybe LLMs can't, but another form of AI will. I hope nobody is interpreting this as "nothing will never be as good as us". I see similar thinking in stories of how humanity got here. Religion has thousands of years adapting to this problem, every time we explain something, the goal post moves. Catholics today accept evolution (or least the church does), but it is the "jump" from monkeys to humans where God is the on…
> Just 5 years ago we didn't have a technology that knows more about everything than even most experts. We still don't have that. LLMs have shown time and time again that they don't know a single thing and are incapable of reasoning.
Re: Position: LLMs Can't Jump
#217Earlier quoted context omitted.
> only worth so much. Worth... quite a lot imo. First, in the slightly objective sense of "What is this place like in real life, under X conditions." But, more subjectively, watching a video of a glacier and imagining the wind/rain/cold doesn't even approach 5% of the intensity and awe of climbing up a mountain yourself to see a seemingly endless expanse of ice, struggling to stand steady because of the wind, shiveri…
There's so much natural wonder in the world in and around your home. It's not that the value of the grand canyon is low in an absolute sense, it's that in a relative sense the gap between "empty numb void" and "hiking at home (plus grand canyon videos)" is far far greater than the gap between "hiking at home (plus grand canyon videos)" and "actual grand canyon".
Sure, no doubt. But I don't really think "video of my local state park + imagining the parts that aren't conveyed over video" is particularly close to the real thing either.
Re: Position: LLMs Can't Jump
#218Earlier quoted context omitted.
Or, ask yourself the following question: If you read every single book there is about The Grand Canyon, and watched every single video and/or documentary about The Grand Canyon, do you believe that you have fully experienced The Grand Canyon? Or do you just have to be there to fully experience it. I dunno. Substitute in whatever you want for "The Grand Canyon". Maybe climbing Mount Everest or walking on the Moon. The…
You have to be there to have a subjective experience of the Grand Canyon, for whatever that's worth. (This is essentially the qualia distinction between us and LLMs, writ large.) But in terms of objective knowledge about the Grand Canyon, reading every article and scholarly work would give you a much better grounding. Unless you're a geologist/ecologist doing actual field research there, actually seeing the Grand Can…
If I'm going to hire a guide, I'm going to hire the guy who spent a week hiking it instead of the guy who spent a week reading about it or watching videos about it.
Re: Position: LLMs Can't Jump
#219Earlier quoted context omitted.
Not at all. The size of an atom was computed for the first time 40 years earlier, in 1865, by Johann Josef Loschmidt, who was an Austrian scientist. Loschmidt had determined the value of what today is called "Avogadro's number", despite the fact that Avogadro had no idea about how one could find the value at that number. The contribution of Avogadro had been the law that ideal gases have the same number of molecules…
I don't understand all this dismissal of Einstein. What you just described is that he published a unified theory of Brownian motion, developed a new way of conceptualizing scientific theories via principles with which he derived special and general relativity, contributed foundational work to quantum mechanics, and more, but no, he was not one of humanities' greatest scientists.
What I have written is not a dismissal of Einstein. On the contrary, I have said that Einstein is the source of a few extremely important discoveries, but those are not the theories cited by most people, who have never read the original work of Einstein but speak about him from hearsay.
As I have written above, when speaking about Einstein, almost nobody mentions the discovery of stimulated emission, which has been far more important than anything else done by Einstein. Einstein did not discover special relativity, the photoelectric effect or Brownian motion. He just provided alternative explanations for them, which did not include any new formula or quantity.
On the other hand, stimulated emission was a surprise, even if it is one of those things that appears to be obvious in retrospect, but for decades nobody who studied the blackbody radiation had noticed that something is missing in the equations.
This discovery eventually lead to the invention of masers and lasers and today there exists almost nothing that incorporates modern technology and which could exist without lasers.
Already for decades, no modern integrated circuits, no CPU and no memory can be fabricated without using lasers. Even many purely mechanical things, but which require great precision, cannot be fabricated without using measurement instruments that depend on lasers.
So the contributions of Einstein are very important, but they are not those that most people associate with him. Besides stimulated emission his most original work was his theory of gravitation, a.k.a. general relativity, but if he had not developed it that might have changed nothing, because Hilbert had developed an equivalent theory almost simultaneously, so had Einstein not published first, Hilbert would have become its discoverer.
After stimulated emission, the most important in practice research of Einstein is the Bose-Einstein statistics. But again, here like for special relativity, the statistics had been discovered by someone else, i.e. by Bose. Einstein has nonetheless the great merit of understanding its value. He demonstrated that the statistics was much more generally applicable than initially believed by Bose and his authority convinced everybody about its importance.
Thus there is no doubt that Einstein is one of the most important physicists of the 20th century. Nevertheless, there are dozens who made similarly important discoveries, e.g. Sommerfeld, Dirac, de Broglie, Schroedinger, Fermi, Rutherford and many others.
Re: Position: LLMs Can't Jump
#220Earlier quoted context omitted.
I get really tired of Einstein being portrayed as if he’s the greatest scientist who ever lived, when he doesn’t even belong in that conversation. If Einstein never existed, the fields he was in were already moving strongly in the directions of his conclusions. If Maxwell or Newton had never existed, the world today would be quite different.
Leading physicists at one time appeared to disagree. Newton and Maxwell do come at #2 and #3 though. http://news.bbc.co.uk/2/hi/science/nature/541840.stm
Einstein is indeed one of the most important physicists of the 20th century, but it is really meaningless to try to establish a hierarchy between physicists, because their works are like open-source programs, each work is based on the works of the predecessors and it would have been impossible to do without those previous works.
In any case, Einstein was good, but Maxwell seems really superior in the sense that he could extract the essential information from the much more confuse sources that were available at his time.
The Treatise of Electricity of Magnetism of Maxwell and his collected papers are very instructive lectures even now and there are fragments of them that remain better than the equivalent fragments in many modern textbooks, despite the huge additional information that we have today.
Most of the original texts written by Einstein are now of interest only for understanding the details of how his thinking and that of the contemporaneous physicists have evolved during those important years, but those texts have little other importance except for history, because later better expositions of the same material have appeared. Nonetheless, it is important to also read Einstein's originals, because in many later books there are some inappropriate interpretations attributed to Einstein, but in reality Einstein had said something else and which was more correct than what some later author misunderstood.