Live data from Hacker News

A recent experience with ChatGPT 5.5 Pro

gowers.wordpress.com

301–310 of 558 posts

Re: A recent experience with ChatGPT 5.5 Pro

#301
post #174
post #166

> "Even though I can motivate it in retrospect, ChatGPT’s idea to use h^2-dissociated sets to control relations of order at most h feels quite ingenious. As far as I can tell, this idea is completely original." The question that keep bothering me is can an LLM generate an idea that is truly novel? How would/could that actually happen? But then that leads to the question - what are we actually doing when we think? Per…

My own take, and it's veering into the Philosophy of Mathematics, but there's a debate about whether Mathematics is "Invented" or "Discovered". If it's "invented", then it requires ingenuity. If it's "discovered", then it was always already there, just waiting for the right connections to be made for it to be uncovered and represented in a way we can understand. Invention requires ingenuity, but discovery does not. S…

Mathematical concepts are invented, but they live in a space of possible (conceivable) mathematical concepts, and we can only invent concepts from that selection of possible concepts. This can be reframed as a process of discovery regarding which conceptions are possible.

Furthermore, the results of theorems aren’t an invention, they are a discovery of what the base assumptions (axioms) logically entail. Finding out which theorems are true and provable is a discovery process. For example, the results of Gödel’s incompleteness theorems were a discovery. They weren’t invented, in the sense that the results couldn’t have been otherwise. We merely could have failed to discover them.

This also holds for physical inventions. You discover a working way to build some functioning mechanism. It’s a process of discovery of what is possible in the physical world.

Whether you portray somethings as a discovery or as an invention is more a matter of degree, a matter of from which angle one is looking at it.

The possible states of an LLM are finitely enumerable. The same likely holds for the possible states and configurations of a human brain, in approximation. Therefore there is only a finite set of possible ideas, thoughts, and conceptualizations an LLM or a human can have, and in principle they could be exhaustively enumerated and thus “discovered”.

Re: A recent experience with ChatGPT 5.5 Pro

#302
An interesting takeaway is that heretofore most of that advances have been not from “invention” but from a breadth of visibility. LLMs have been able to be “creative” because of the volume of work that they cover and can draw lines and associations between, not in discovering things that did not exist previously (though an argument can be made that something like AlphaFold was “discovering” and “intuiting” associations that were not explicit anywhere previously, uniquely found by the AI… but I’d argue back something about the bitter lesson and we’d go on for more than a few threads).

Somewhat ironic then, to not make this more explicit in an article about solving a combinatorial problem.

Re: A recent experience with ChatGPT 5.5 Pro

#303

Earlier quoted context omitted.

A depressing thought that all that work is just so you can "command AIs better"

It could happen than the AI, in a near future, is not something external but just a part of your brain, so you retain the glory.

Why stop there? Why not let AI take over all functions, Whispering Earring (https://gwern.net/doc/fiction/science-fiction/2012-10-03-yva... for anyone who hasn't read it) style?

Re: A recent experience with ChatGPT 5.5 Pro

#304
post #186
post #174

Earlier quoted context omitted.

My own take, and it's veering into the Philosophy of Mathematics, but there's a debate about whether Mathematics is "Invented" or "Discovered". If it's "invented", then it requires ingenuity. If it's "discovered", then it was always already there, just waiting for the right connections to be made for it to be uncovered and represented in a way we can understand. Invention requires ingenuity, but discovery does not. S…

I like this distinction, but it would then seem the only 'invention' would be the axioms of your mathematics. There exists numbers (natural, imaginary...), there exist shapes (a point, a line...). All the work from that point on could be 'discovered'. I agree that I don't see LLMs inventing in this way. But again, it raised the question - what are our brains doing when we 'invent' something?

Well, take any invention you like, and let's break it down.

Somebody at some point, "invented" the idea that the earth was round. Before that, the obvious "just look around you" answer would've been, duh of course the ground is flat. But we know the earth has always been round, even if humans couldn't appreciate it for hundreds of thousands of years (I don't count the pre-history before homo sapiens). So we "invented" some fields of science and the mental models / abstractions that allowed us to conceptualize what a round earth could mean and how to measure it, but we didn't invent the roundness itself -- that was always reality, and we just lacked both the thoughts and the tools to conceptualize it (until later).

Now you might say, well that is a category of "simple" physical observations. The earth is naturally round all the time and doesn't take any extra human effort to make it so (it took some effort to imagine that it could be and to find ways to measure/prove it). But what about say -- semiconductors, NVIDIA GPUs, that sort of thing? It's not like semiconductors grow on trees and we just need to find them and learn how to consume/use them... isn't that a better example of "true invention"?

Sure, I could see that. But I guess my POV would be that, the invention of the latest AI chip, or the first semiconductor, or the first vacuum tube, or whatever came before, all laddered largely incrementally on "discoveries" that were then cleverly tweaked or reapplied, so that what appears to be "true invention" is usually/more-often just another chain in a long chain of "discoveries" that led up to it. I grant you that some of what appears in hindsight to be continuous progress, really is built on small discontinuous "leaps", but I don't think that breaks the argument (strengthens it in fact, IMO). You wouldn't have semiconductors today, unless Faraday (or somebody like him) discovered that silver sulfide resistance decreases with heat, and that is more like one of those physical properties that reality has always had (much like, earth was always round, we just didn't know it at first).

So in that sense, I feel this becomes almost like an "evolution vs intelligent design" debate -- some people look at the complexity and miracle that is the human eye or the human brain, and they insist there must have been an intelligent designer, because surely no random chaotic biological process could have produced something so wonderful... And yet, I think the scientific evidence largely shows that, indeed that is what happened, just random chance + evolutionary-pressure was all you really need (plus billions of years). So if you can accept that analogical framing for a minute, then I would posit that "invention"-adherents are really making something like an intelligent design argument, vs "discovery"-adherents are saying that evolution (in an artificial sense, with the artificial selection pressures of scientific research, of capitalism, etc., and compressed into centuries or decades, not millions or billions of years) is sufficient to derive miraculous-seeming results. The little discontinuous leaps along the way, are kind of like the random mutations of genes that happen to confer an advantage -- maybe we can say that we are more intentional about seeking those leaps out, or maybe we are just right-place/right-time lucky (e.g. thinking about penicillin and the random petri dish left out).

Perhaps once (or if) there is the sort of leap that breaks us out from a Type I to a Type II+ Kardashev civilization, maybe then I would grant you something needed to be "invented" that couldn't be based on a line of "discoveries". Or maybe not, maybe it will just be another semi-random discovery.

Re: A recent experience with ChatGPT 5.5 Pro

#305

Earlier quoted context omitted.

This could be right for the current architecture of LLMs, but you can come up with specialized large language models that can more efficiently use tokens for a specific subset of problems by encoding the information differently ( https://www.nature.com/articles/d41586-024-03214-7 ). So if instead of text we come up with a different representation for mathematical or physical problems, that could both improve the qual…

> This could be right for the current architecture of LLMs, but you can come up with specialized large language models that can more efficiently use tokens for a specific subset of problems by encoding the information differently. That's precisely what happens on the bad side of a S curve.

Progress don't stop however, and the S curve resets, because then you are optimizing a new architecture.

Re: A recent experience with ChatGPT 5.5 Pro

#306
post #187
post #173

>> but it was definitely a non-trivial extension of those ideas, and for a PhD student to find that extension it would be necessary to invest quite a bit of time digesting Isaac’s paper The "non-trivial" is for human abilities. The weights lifted by a crane are also "non-trivial". People keep getting amazed at machine's abilities. Just like a radio telescope can see things humans can't, microscope can see the detail…

Too many people are wrapped around the ego axle thinking (assuming) their ideas are both them and somehow unique and special. It usually takes dissolving that, often through difficult experiences, before they can see it as a machine, something that could be separated from them.

I think the more pressing issue is that there isn't really much space left for humans in the economy if thinking can also be automated.

Re: A recent experience with ChatGPT 5.5 Pro

#307

Earlier quoted context omitted.

This could be right for the current architecture of LLMs, but you can come up with specialized large language models that can more efficiently use tokens for a specific subset of problems by encoding the information differently ( https://www.nature.com/articles/d41586-024-03214-7 ). So if instead of text we come up with a different representation for mathematical or physical problems, that could both improve the qual…

> So if instead of text we come up with a different representation for mathematical or physical problems, that could both improve But then, wouldn't we first have to translate all of our current math and physics knowledge into that new representation in order to be able to train a model on it? Looks like a tremendous amount of work to me.

Yes, but by then you already have general LLMs capable of helping with the work. And even if you didn't, if that's what it would take to advance research in these fields, that would be a justifiable effort.

Re: A recent experience with ChatGPT 5.5 Pro

#308
post #238

Earlier quoted context omitted.

In the sense that the incremental improvements in capabilities that we've been seeing in recent models seem to taking exponentially growing amounts of compute to achieve.

But they don't? Mythos is a 10T model. Opus is a 5T model. That's not an exponentially growing amount of compute but it is achieving exponential improvements (eg from Mozilla: https://blog.mozilla.org/en/privacy-security/ai-security-zer... )

> Mythos

Ah yes, the marketing model that's ostensibly so powerful us mere mortals aren't allowed to use it. It's certainly led to exponential hype and speculation.

Re: A recent experience with ChatGPT 5.5 Pro

#309

Earlier quoted context omitted.

Nobody is releasing NEW models

What? DeepSeekV3 just came out and is incredible for the price. Mythos is also half-released.

Until you or I can actually use Mythos in Claude without an nda or other strings attached, Mythos is not released and is just an effective marketing tool for Anthropic.

Re: A recent experience with ChatGPT 5.5 Pro

#310
Gowers has always been a proponent of Lean (naturally). He receives funding from the "AI for Math" fund, which is sponsored by a fund that is a front organization for venture capitalists:

https://www.renaissancephilanthropy.org/

The "brighter future" of course is that everyone is redundant and all capital is further concentrated.

It is always Gowers, Tao and Lichtman (math.ínc startup) who are pushing these technologies.

Post reply on HN