Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ten advances in mathematics and theoretical computer science

openai.com

Post reply on HN
Print view View original

211–220 of 1000 posts

Re: Ten advances in mathematics and theoretical computer science

#211

Post by c7b »

aabhay wrote: post #6

My main gripe here is the lack of transparency around the total experiment and construction. I doubt that they simply pointed their model at these ten specific problems alone and gave the model one shot; therefore the $2000 number could be completely misleading, similar to P-value hacking by not disclosing the total experimental setup. I want to know: 1. How many total problems were given to the model, and what perce…

I believe we're seeing a new kind of mathematics that will require completely new formats for publication, a bit similar to those used in experimental sciences. AI-powered mathematics should be fully reproducible, so it's the authors' responsibility to disclose the exact model type, inference settings/seeds and the full prompt history leading to the result. Of course that would ideally require open weights models.

It's not just about requiring to disclose AI use. AI-powered mathematics is a completely valid discipline that doesn't need to be shy, but it should develop its own publication culture.

Re: Ten advances in mathematics and theoretical computer science

#212

Post by slashdave »

ultimatefan1 wrote: post #118

one of the early premises of how ai takeoff would go was that a system that could solve open problems in advanced mathematics would also discover novel advances in math and computer science that directly unlock drastically better software performance. we are seeing frontier level math breakthroughs (ie performance that would put it in the top 100 or 1000 mathematicians in the world if it were a human, meaning top .00…

> we are also seeing incredible advances in software performance

Incredible?

> open ai announced like 15% improvement by fixing gpu kernel issue

That is... ordinary software optimization.

Re: Ten advances in mathematics and theoretical computer science

#213

Post by kingstnap »

emil-lp wrote: post #138
kingstnap wrote: post #114

Earlier quoted context omitted.

Why would you factor in salary unless they had to baby it through. You would only count the hours for setting up the harness and prompt and checking the result. Training the model is going to be amortized over other uses.

> Why would you factor in salary Say that it turned out that the total cost of the proof of the Erdős unit-distance conjecture was $50 million. Then the question really becomes: yes, these models are capable of proving important mathematical results, but at a very high cost. Is it worth it? If a mathematician applied for a research grant of $50M USD for proving the same thing, they would have been laughed out of the…

I didn't argue that knowing the total cost is uninteresting. What I was saying is that realistically the total cost is:

Hours needed for prompt + Hours needed to check result + API costs.

You don't say "well let's add together the total yearly compensation of all the engineers and mathematicians at OpenAI that were involved" and throw that into the total cost. That's simply nonsense accounting.

The actual comparison you are making is some university researcher weighing between getting a grad student (several tens of thousands of dollars) vs typing up a prompt and sending a request to OpenAI for inference (as mentioned in the article, around $2000 in API and maybe a few hours for the prompt and harness).

Re: Ten advances in mathematics and theoretical computer science

#214

Post by slashdave »

artninja1988 wrote: post #115

Now that we've seen AI produce a fair number of proofs (and disproofs), I'm curious when we'll start seeing it build genuinely novel theory. Does anyone have predictions on when and how we'll get there and will it take new architectures/ training paradigms, or is the current approach enough?

It will not happen with existing LLM techniques.

Re: Ten advances in mathematics and theoretical computer science

#215

Post by slashdave »

laichzeit0 wrote: post #140
artninja1988 wrote: post #115

Now that we've seen AI produce a fair number of proofs (and disproofs), I'm curious when we'll start seeing it build genuinely novel theory. Does anyone have predictions on when and how we'll get there and will it take new architectures/ training paradigms, or is the current approach enough?

I’m personally hoping for the next big AI gangbanger to be theoretical physics. Boy does that field need a good reshuffle. I think when any novel mathematical theory can be done by AI you’ll see simultaneously theoretical physics getting wrecked as hard as pure math is. At that point we might see new physics or paradigm shifting technology emerging.

What? No. Frontier physics is experiment driven.

Re: Ten advances in mathematics and theoretical computer science

#216

Post by slashdave »

christofosho wrote: post #134

I would love more time and money put into real-world problems by these companies. Climate, food insecurity, pollution, technology for convenience and/or to help people have a higher quality of life. I'm sure they must do some of this type of work, right?

Seriously?

We already know how to solve all of these issues. What we lack is collective political will.

Re: Ten advances in mathematics and theoretical computer science

#217

Post by naasking »

Delk wrote: post #77
naasking wrote: post #19

Earlier quoted context omitted.

> AI has no self-awareness What is your mechanistic model of self awareness that yields this conclusion? > It's a tool Does your model suggest that tools can't have self awareness?

I honestly don't think a language model is enough for self-awareness, regardless of the exact model of awareness. A language model (or an image model or whatever) cannot even be sentient, and I think sentience is a prerequisite for awareness. Even if we express a lot of our subjective experience with words, the language is just a symbolic representation of those experiences. The qualia themselves, even those that are…

> The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.

There is no objective evidence of qualia. All evidence of qualia are vocal or other expressions of belief in qualia. Perceptions clearly exist and are observable, subjective experience and qualia, not so much.

> I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.

If your objection is to models based on "symbolic level of language" which you think lack semantic understanding of, say, trees, you should ask yourself how our brain, based on physics which also lacks any semantic category for trees, can somehow develop a semantic understanding of trees. All of these appeals to differences with the brain never seem to acknowledge that fundamentally, the brain has the same explanatory gap with physics.

> But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience

This assumes a lot. It seems very possible to me that intelligence inherently develops a map of natural categories (natural kinds), and language naturally develops around such categorical understanding. Semantics are then fundamentally the network of associations between categories, eg. there is no fundamental difference between symbols and semantics, and the latter cam be inferred from the former, and that's exactly what LLMs do, and why the semantic maps between different languages are so similar and how they can translate between languages.

Re: Ten advances in mathematics and theoretical computer science

#218

Post by naasking »

Delk wrote: post #193
woeirua wrote: post #146

Earlier quoted context omitted.

So… your model is 100% vibes based. Got it.

I wasn't trying to give a model. The point was that I don't think it's necessary to give one. You didn't address any of what I wrote, let alone provide any counterarguments. Which part of what I wrote do you think was wrong?

Making definitive claims about whether LLMs do or do not have specific properties absolutely does require precise definitions of those properties that can be used to evaluate those questions. Merely hand waving that LLMs didn't undergo the same evolutionary process is not a definitive argument.

For example, the Turing machines and the lambda calculus don't look anything alike, but they are fundamentally interconvertible, and so in a real sense they are fundamentally equivalent. Without a model, all of your arguments are completely unconvincing for exactly the same reasons, eg. that there may exist many paths to fundamentally equivalent ends.

Re: Ten advances in mathematics and theoretical computer science

#219

Post by naasking »

perching_aix wrote: post #51
naasking wrote: post #19

Earlier quoted context omitted.

> AI has no self-awareness What is your mechanistic model of self awareness that yields this conclusion? > It's a tool Does your model suggest that tools can't have self awareness?

Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to, and influences future choices. This implies statefulness, which models are intentionally not at inference time (*). (*) Even if we hack around this and just do the usual trick of simply laundering statefulness to a higher level, in this case the context window being fed in,…

> Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to

I don't see why an LLM could not have a sense of identity or personality while it's evaluating a specific prompt, or even change self awareness while evaluating a prompt since many outputs model a back and forth conversation. My point is that without a mechanistic model of what "self awareness" means, we have no way of truly evaluating such questions, we're just hand waving vague intuitions about what it could mean.

Re: Ten advances in mathematics and theoretical computer science

#220

Post by simianwords »

esperent wrote: post #164
simianwords wrote: post #141

Earlier quoted context omitted.

Would this be your reaction if OpenAI also solved millennium problems? The point we are trying to make is that the significance of this news is much larger than the skepticism you are providing.

It would be my reaction if we're discussing a blog post from OpenAI, yes. I would be looking at it extremely critically, wondering what they're misrepresenting to make it look cheaper, easier, and why they're trying to make it look like only their model could possibly do this. Look at their recent claims about their model "escaping" - there was literally a Guardian article calling them out for being hyperbolic! Again…

Hmm. But this level of skepticism looks performative and seems to serve as a signalling thing rather than a functional thing. You do you though. If OpenAI solves the millenial problems, my skepticism will only be restricted to the correctness of proof. Not that it was "marketing" haha
Post reply on HN