Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

441–450 of 844 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#441

Post by dekhn »

randomblock1 wrote: post #363
OneManyNone wrote: post #325

Earlier quoted context omitted.

“ On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems.” - https://openai.com/index/navier-stokes-solution/ They do not explicitly admit to knowing about NS specifically, but are e…

So then they DIDN'T "learn the secret to cracking the problem". They simply knew that part of the problem was solved. Knowing a problem can be solved and knowing the solution are not the same thing.

The claim that OpenAI somehow used the mathematicians' ideas to leapfrog them seems unsupported at this time and IMHO it was irresponsible to bring it up because credulous people will immediately believe that narrative.

And from my perspective, if some math folks typing in a few questions to OpenAI provides sufficient training data for OpenAI to solve a big problem... that's amazing! A few conversations/prompts out of the billions that OpenAI trains on lead to this result- that means there is an awful lot of low-hanging fruit that could be exploited cheaply.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#442

Post by madaxe_again »

calf wrote: post #403
madaxe_again wrote: post #110

Earlier quoted context omitted.

Actually, my undergraduate degree was physics and philosophy. And yes, synthesis is synthesis whether a human, a machine, or a duck does it, and people prompt one another all the time - “have you thought about trying X?” Or “I need the TPS report by EOB”. I suppose my underlying point is that human cognition is not the unique and beautiful thing that we anthropocentrically suppose it to be - it is a physical process,…

Clearly your degrees did not make you immune from fallacies and simplistic reductions. "Synthesis" is obviously of different kinds. A duck has a different level of intelligence than a human. We do not say both are "just doing synthesis". So the question is how can you be so disingenuous about such terminology? Answer, you are relying on a classic form of scientistic reductivism. The fact that intelligence is physical…

“obviously of different kinds”

What’s your basis for that “obviously”? You have a unique insight of the phenomenology of duck-ness? You can prove that your consciousness is somehow real, somehow different? A duck synthesises with its cognition, or it would be incapable of, well, anything. Synthesis is purely the process of the integration of inputs into outputs - ie behaviour, language.

Here’s an article on a paper on duck synthesis:

https://www.pbs.org/newshour/science/ducklings-make-way-abst...

“objectively different levels of computational sophistication”

Says who? We still have a very poor understanding of how cognition works in animals, humans included. For all we know ducks have rich inner lives - a remarkable amount can be achieved with a very small neurone count - cf. insects. Can you coordinate flight? Can you echolocate? Are you less intelligent because you cannot?

“equivocation of "mentorship" as "prompting" and fallacious variants thereof”

You are arguing semantics. Take Harry Nyquist. He sent people down new paths with insightful questions. You could call this mentorship if you choose, I could call it prompting, but this splits hairs. The core idea is that a novel input can produce a novel output, that synthesis can be induced through guided and deliberate external input.

I invoked credentials only in response to the previous derogatory comments about my cognition - which may or may not exist, anyway.

As to religiosity - the idea that human cognition is somehow unique and special and impossible to replicate, which is the prevailing argument in this comment tree is religious, and anthropocentrism of the highest order. I apologise for accusing you of it - I was evidently wrong - I had mistaken you for a previous poster.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#443

Post by brulard »

WD-42 wrote: post #411
sashank_1509 wrote: post #266

Both things can be true: 1. OpenAI when using your chats in pretraining is improving its model’s intuition. The model parameter size is massive, and while the data is OOM larger it is plausible that model remembers stuff about chats that improves its latent representation. 2. During RL on verifiable math and massive compute, the model discovers techniques and connections to solve math problems that are superhuman and…

If they have solved hundreds of open problems in math, why are they publishing results for the ones other mathematicians happen to be working on at the same time? Why not the others?

You think other mathematicians are currently working on very little subset of relatively low-hanging fruit problems?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#444

Post by crostlybostly »

pfortuny wrote: post #414
famouswaffles wrote: post #371

Earlier quoted context omitted.

OpenAI have come out and said: >The Wednesday evening statement from OpenAI was more emphatic: “We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.” >The statement added, “After investigating, we can say with full confidence that no user inputs past July 3rd could have influenced this system in any way…

Apart from the well-known dubious position of OpenAI wrt truth, the prompts/inputs do mot include the outputs. You can train on a sequence of outputs. In the end, OpenAI outputs are OpenAI's property. You can learn a lot from a single side of a conversation.

But using the outputs to train would make their statement false, since they are influenced by the inputs

Re: More questions about whether researchers can trust OpenAI with unpublished math

#445

Post by jcims »

tomjakubowski wrote: post #401
jcims wrote: post #392

Earlier quoted context omitted.

>Quote: "The Overhang consists of the unrealized capital gains of past mathematical creativity, the latent value from connecting the dots in the existing corpus. It is a dividend of canonization. Mathematician X states problem A, mathematician Y crafts concept B, then mathematician Z notices that B trivially solves A and “captures” the social reward. I've made an entire career out of being 'jack of all trades, master…

A college advisor I had 20 years ago was a firm believer that interdisciplinarity was the future, that generalist skills and the ability to make connections between different fields would be paramount in advancing science. I suppose he was right in the big picture, even if the career prospects for human generalists aren't looking so rosy.

I'm actually still quite bullish on generalists. Specialists advance every front but build the supply lines between them.

In favor of the generalist, I think AI is also quite limited in its scope of how it generalizes. I'm mowing through hundreds of mythos-generated security findings right now for work and while it's amazing that it can build an exploit chain 20 steps deep, it's completely lacking in all of the external layers that render it's speculation moot.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#447

Post by sensanaty »

dekhn wrote: post #436
bena wrote: post #376

Earlier quoted context omitted.

This is literally "We have investigated ourselves and found no wrongdoing" Why should we trust them?

Reputational risk- if they lie about this and get caught, it will have billion dollar implications for their business.

Every single thing these companies do is dishonest and every word that comes out of the lips of these company execs is a lie, what fantasy land are you living in in which anyone with any amount of power gets punished for their lies?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#448

Post by ForHackernews »

dgellow wrote: post #286
sashank_1509 wrote: post #266

Both things can be true: 1. OpenAI when using your chats in pretraining is improving its model’s intuition. The model parameter size is massive, and while the data is OOM larger it is plausible that model remembers stuff about chats that improves its latent representation. 2. During RL on verifiable math and massive compute, the model discovers techniques and connections to solve math problems that are superhuman and…

I feel that we don’t praise Lean enough. AFAIU it’s what enables LLMs to brute force those problems

How long until we find out that some AI has quietly buried an exploit in Lean to cheat at proofs?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#449

Post by jimmydddd »

nezi wrote: post #412

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model…

So, at my company (and most companies I think), we use confidential in-house versions of the AI software. We don't want any confidential information leaking into the public realm. Are these scientists doing that, or are they just using the public version of the software?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#450

Post by ForHackernews »

pixl97 wrote: post #311
ozgung wrote: post #293

Earlier quoted context omitted.

If your rumor is true, what we are witnessing is a giant paradigm shift rather than individual incidents. Mathematicians were the first victims of super-intelligence. Of course it’s not an endless source. They had to burn millions of dollars to solve a single problem.

>They had to burn millions of dollars to solve a single problem I'd like to adjust that to "They had to burn a lot of energy (create a lot of entropy) to solve a single problem. As we go into the super-intelligence age the current paradigm of money as humans understand it may break at some point. For example to a paperclip-maximizer money at best is a short term instrumental goal, hard power of matter conversion mach…

I'd wager a fair chunk of my money that money breaks OpenAI before OpenAI breaks money.
Post reply on HN