401–410 of 1001 posts
Earlier quoted context omitted.
We have had LLMs for much longer than 3 years.
No, we haven't, for any reasonable definition of L.
Earlier quoted context omitted.
I like to think of it as: Imagine every bit of human knowledge as a discrete point within some large high dimensional space of knowledge. You can draw a big convex hull around every single point of human knowledge in a space. A LLM, being trained within this convex hull, can interpolate between any set of existing discrete points in this hull to arrive at a point which is new, but still inside of the hull. Then there…
A lot of the space outside of the convex hull is just untried things. You can brute-force trying random things and checking the result and eventually learn something new. With a better heuristic, you can make better guesses and learn new things much more efficiently. There’s no reason to believe that kind of guess-and-check is outside of the reach of LLMs, or that most of our new discoveries are not found the same wa…
Earlier quoted context omitted.
> What is the difference between what LLM's do and "true" understanding? The thing where you can understand the meaning of this sentence without first compiling a statistical representation of a 10 trillion line corpus of training data. Unless you're an NPC of course.
I mean brains get a lot of training data too in order to understand language. I don't think you provided a relevant difference. Or rather, maybe I don't understand what you mean :)
Earlier quoted context omitted.
The map is not the territory.
I don't know what you're even trying to argue here. We're not comparing math to reality (though there's a strong argument to be made that reality has a structure that is mathematical in nature - structural realism didn't die a scientific philosophy just because someone came up with a pithy saying), we're talking about if math is discovered or invented. Most mathematicians would argue both - math is a language, we hav…
It seems that addition (for instance) was "created" long before us.
On the other hand, it seems highly unlikely that a civilization similar to ours could "invent" an essentially different kind of mathematics (or physics, etc.)
For anyone using LLMs heavily for coding, this shouldn't be too surprising. It was just a matter of time. Mathematicians make new discoveries by building and applying mathematical tools in new ways. It is tons of iterative work, following hunches and exploring connections. While true that LLMs can't truly "make discoveries" since they have no sense of what that would mean, they can Monte Carlo every mathematical tool…
> the result has no meaning without the human intent behind the objective, human understanding to value the new pathway the AI used (more valuable than the result itself, by far) and the mathematical language (built by humans) to explore the concept. Isn't this just anthropocentrism? Why is understanding only valid if a human does it? Why is knowledge only for humans? If another species resolved the contradictions be…
People saw birds fly for all of human history, but it was only recently that humans were able to make something fly and understand why. Once we understood, we were able to do amazing things, but before that, the millions of birds able to fly were of no help beyond inspiration for the dream.
The proof brings unexpected, sophisticated ideas from algebraic number theory to bear on an elementary geometric question. The more I read about these achievements the more I get a feeling that a lot of the power of these models comes from having prior knowledge on every possible field and having zero problems transferring to new domains. To me the potential beauty of this is that these tools might help us break thro…
As we're becoming hyper specialised, they become an invaluable tool to merge the horizon in, so to speak.
Timothy Gowers' tweet about this: "If you are a mathematician, then you may want to make sure you are sitting down before reading futher.". woah.
Unfortunately Gowers has taken Tao's lead on this one. Gowers has one of my favourite video series about how he approaches a problem he is unfamiliar with: https://www.youtube.com/watch?v=byjhpzEoXFs It is disheartening to see him jump into this GenAI puffery. I hope these GenAI labs are paying Tao handsomely for legitimizing their slop, but more likely he's feeling pressure from his University to promote and work wi…
Earlier quoted context omitted.
What's happening is the verbal/linguistic equivalent of the invention of calculus. No intellectual field will ever be the same again. Who wouldn't find that exciting, and want to experience it?
People who enjoy thinking. Ya know, the "intellectual" part.