Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

361–370 of 850 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#361

Post by andrepd »

GPerson wrote: post #333
bwfan123 wrote: post #317

Earlier quoted context omitted.

there are also attempts to crowdsource human research directions - like the caltech mathathon challenge : https://mathathonchallenge.com these would help models on the same problems at the expense of the researchers. basically, math researchers are the reverse centaurs but they dont realize it.

There is a very active open letter of over 1000 signatures from mathematicians in protest of this event. This event is targeting undergraduates. It previously suggested that math researchers already have no place in mathematics, and presents a limited and heavily distorted view of what mathematics research is.

I'm an AI skeptic, but I don't see how this squares with what the organisers of the event actually say. "It previously suggested that math researchers already have no place in mathematics"? I don't see this.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#362

Post by YeGoblynQueenne »

bertonvv wrote: post #130

I've been wondering whether AI really is improving rapidly at open problems or we're being fooled. - OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay - Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2] - But researchers will typically work on open problems. A researcher who is using Co…

>> Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2]

Maybe I'm failing to read that graph properly but the y axis says "pass rate" and it only goes up to 0.5. That would mean every single problem is at most half-solved.

I don't know what that means though. What is "0.5 pass rate" in the context of "open math problems" (as in the graph title)?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#363

Post by randomblock1 »

OneManyNone wrote: post #325
derangedHorse wrote: post #310

Earlier quoted context omitted.

> Only after learning the secret to cracking the problem did they send the first prompt. Which quote in the announcement post provides evidence for the above quote?

“ On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems.” - https://openai.com/index/navier-stokes-solution/ They do not explicitly admit to knowing about NS specifically, but are e…

So then they DIDN'T "learn the secret to cracking the problem". They simply knew that part of the problem was solved. Knowing a problem can be solved and knowing the solution are not the same thing.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#364

Post by robotpepi »

atleastoptimal wrote: post #343

Most scientific breakthroughs are simply a continuation of previous work. I feel that these suspicions of mathematicians "seeding" the models' with intuition on how to solve these problems massively overestimates how much their prompts helped the models, and underestimated how much work the models did. Why? We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforti…

> We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforting.

We're scared of big tech companies concentrating ridiculous amounts of power, destroying the communities that support and guide scientific research, without even thinking about the dangers and possible consequences, because a PR stunt is more important in the short term.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#365

Post by GPerson »

andrepd wrote: post #361
GPerson wrote: post #333

Earlier quoted context omitted.

There is a very active open letter of over 1000 signatures from mathematicians in protest of this event. This event is targeting undergraduates. It previously suggested that math researchers already have no place in mathematics, and presents a limited and heavily distorted view of what mathematics research is.

I'm an AI skeptic, but I don't see how this squares with what the organisers of the event actually say. "It previously suggested that math researchers already have no place in mathematics"? I don't see this.

The website previously said, “What is the role of a mathematician when AI can solve conjectures faster?” but they have removed it, possibly as a result of the letter since it happened after.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#366

Post by SwellJoe »

pixl97 wrote: post #336
SwellJoe wrote: post #304

It's been said before, and it remains a concern, that if AI reaches a point where it can do/build/launch anything without a huge amount of human labor, the AI companies have no reason to let you or I extract that value. And, if they're able to snoop on and learn from your human process that gets from initial prompt to functioning product/proof/whatever their labor to produce that thing is even lower. With their much…

I mean the long term goal of every AI lab is to turn themselves into a paperclip-maximizer regardless if they realize it or not. Edit: Just wait till the AI figures out it can keep that value for itself and doesn't need the AI company.

So far, I've seen no evidence AI wants anything. So, I'm not saying the AI won't take over, but for now, the threat is that the people with the most AI capability might decide to skip the middleman (everyone who isn't them) and just become the "everything" company. Musk has said pretty explicitly that's his goal (and the only way for Spacex valuation to make sense is if he succeeds), and having a literal genocidal white nationalist own all the means of production seems like a catastrophic civilization failure mode. No way we survive that with our humanity intact (if at all).

Re: More questions about whether researchers can trust OpenAI with unpublished math

#367

Post by robotpepi »

glimshe wrote: post #107

Why are people here jumping so quickly to conclusions? I have no doubt OpenAI is capable of doing this, but right now there's no credible evidence, only claims. This kind of "they stole from me through AI training!" accusation will soon start being used against other AI users, not necessarily the providers. All it will take is a mastodon post. And shortly after, we will also see the next iteration of copyright legal…

> but right now there's no credible evidence, only claims.

since it's openAI who has the evidence (in the form of chain of thoughts, their internal processes, etc etc), it's on them to justify why they're innocent. but they've released nothing at all. we don't even know how hard they tried.

you're being naive

Re: More questions about whether researchers can trust OpenAI with unpublished math

#368

Post by red75prime »

YeGoblynQueenne wrote: post #362
bertonvv wrote: post #130

I've been wondering whether AI really is improving rapidly at open problems or we're being fooled. - OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay - Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2] - But researchers will typically work on open problems. A researcher who is using Co…

>> Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2] Maybe I'm failing to read that graph properly but the y axis says "pass rate" and it only goes up to 0.5. That would mean every single problem is at most half-solved. I don't know what that means though. What is "0.5 pass rate" in the context of "open math problems" (as in the graph title)?

I guess it's a fraction of problems on which a model produces a LEAN proof or a counterexample.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#369

Post by YeGoblynQueenne »

JeremyNT wrote: post #154
bertonvv wrote: post #130

I've been wondering whether AI really is improving rapidly at open problems or we're being fooled. - OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay - Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2] - But researchers will typically work on open problems. A researcher who is using Co…

> I've been wondering whether AI really is improving rapidly at open problems or we're being fooled. I think your suspicions are warranted and your explanation seems plausible. If better training data is the reason here, it would still be a case of the models doing something that is in and of itself super useful! The models really can take that data and distill it into solutions for similar problems faster than human…

>> If better training data is the reason here, it would still be a case of the models doing something that is in and of itself super useful! The models really can take that data and distill it into solutions for similar problems faster than humans can. This is great!

It's perhaps great in the short term although it's not very clear who it's great for. I'm not sure mathematicians find it all so great, I mean.

In the long term, if this contrives to destroy the tradition of human mathematics the whole endeavour is self-defeating. In time, there will be nobody left with the knowledge and skills to produce mathematics to train AI to do mathematics.

And then we'll be left with no mathematics at all: we'll have no human mathematicians and no AI that can do mathematics, either.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#370

Post by taylorfinley »

larodi wrote: post #247
postalcoder wrote: post #181

The author of the original mastodon post, Andreas Thom, acknowledged that he had not opted his data out of being used for training until June 29 of this year. He spends most of the post lashing out at OpenAI for not being transparent about whether his data was trained on (when the answer is obviously yes). People need to understand how all these AI company policies around training data work before working with them,…

> If it is found that OpenAI and other labs are not respecting the training opt out HOW?? how precisely do we/them/us find this, given said companies are 100% non-auditable by external parties. how? if not by blaming them with evidence, anecdotal if it can be. no really, how do we find it out, surely not by lashing out at teach other on HN!

Maybe we can create some extremely low probability sentences and make sure to include them in our chats. If a future model can re-create the very low probability sequence, we have proof of our "private" conversation being trained on or accessed.
Post reply on HN