Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

801–810 of 849 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#801

Post by twobitshifter »

jasonfarnon wrote: post #753
Agentlien wrote: post #733

Earlier quoted context omitted.

I think this move by OpenAI is crazy. At best, if all unconfirmed accusations are unfounded, they still heard a rumour that someone had solved a huge million dollar problem and was about to make a name for themselves. Then, they decided this was a good opportunity to pour millions of dollars into trying to snag the glory while the researchers were busy cleaning up their notes and polishing the announcement. That stil…

Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first? Often knowing of the mere existence of a solution carries a lot of information--you would know the problem is accessible, you would expect clues in recent progress (the two Spanish researchers in this case), you would probably have a sense if the solution is a counterexample or positiv…

According to Buckmaster, the prompt used on the AIs was based on his approach and solution that was unpublished. So they were starting from 90% of the way there.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#802

Post by madaxe_again »

YeGoblynQueenne wrote: post #783
ImPostingOnHN wrote: post #429

Earlier quoted context omitted.

You are simply incorrect. It is not a fallacy of that type, or any other type, because the words do, in fact, mean the same thing, as multiple people have pointed out here. Whether referring to chatbots or people, "prompt" means "to move to action". If you have some reliable source supporting your unilateral claims that "prompt" does not mean this, please share. Otherwise, the consensus seems to be contrary to your c…

Why do I need a source? An LLM prompt does not "move to action", because an LLM does not act. People act, animals act, software doesn't act. Acting implies volition and volition implies cognition and if you think that LLMs have those things then you're the one who should provide a source for your claim.

How about an LLM connected to a robotic arm. What then?

As for cognition - can you prove that you possess it, to an external observer? Could you, confined to a box through which you can only communicate through textual messages, prove that you are thinking, and not just responding through a mechanistic process?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#803

Post by fwlr »

bertonvv wrote: post #130

I've been wondering whether AI really is improving rapidly at open problems or we're being fooled. - OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay - Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2] - But researchers will typically work on open problems. A researcher who is using Co…

The relentless progress towards saturation of benchmarks is, I suspect, at least partly a similar story. Whatever holdout questions are used to evaluate GPT-x will be in the GPT-x conversation logs, and therefore in the training set for GPT-x+1.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#804

Post by samuelknight »

fwlr wrote: post #803
bertonvv wrote: post #130

I've been wondering whether AI really is improving rapidly at open problems or we're being fooled. - OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay - Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2] - But researchers will typically work on open problems. A researcher who is using Co…

The relentless progress towards saturation of benchmarks is, I suspect, at least partly a similar story. Whatever holdout questions are used to evaluate GPT-x will be in the GPT-x conversation logs, and therefore in the training set for GPT-x+1.

I can't think of better invention than one that can saturate a benchmark of every interesting problem.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#805

Post by majkinetor »

nomel wrote: post #652
jcims wrote: post #392

Earlier quoted context omitted.

>Quote: "The Overhang consists of the unrealized capital gains of past mathematical creativity, the latent value from connecting the dots in the existing corpus. It is a dividend of canonization. Mathematician X states problem A, mathematician Y crafts concept B, then mathematician Z notices that B trivially solves A and “captures” the social reward. I've made an entire career out of being 'jack of all trades, master…

How do you thrive in an environment of specialists? That's is the problem I seem to have. I'm spread a little across a few of the domains involved with what I do. Because of that, I have a bit more insight, so am very often the person pointing out relatively fundamental problems , usually caused by either not understanding the problems from a "first principles" perspective, resulting in, or being caused by, categoric…

Yes, advice.

Eventually, you will be seen as a party breaker, and all that will be seen as negative. You are the bringer of the bad news; nobody likes that, no matter how reasonable you are. Shitty products are the norm when profit is in question and doing what is right is not and will be in most places taken as inability to be a valuable team member. There might be some slight chance that you will be in good environment, but even that is usually temporary as business owners, goals and incentives change.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#806

Post by ModernMech »

eru wrote: post #705
ZYbCRq22HbJ2y7 wrote: post #645

Earlier quoted context omitted.

> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well. Because the training data is millions of hours human efforts being distilled into a cascading hierarchy of enrichment by interested parties without providing attribution or compensation?

I don't see the cascading hierarchy of enrichment. I mean, they are certainly trying, but so far there's too much competition so the surplus mostly goes to customers.

> I don't see the cascading hierarchy of enrichment.

If there wasn't a hierarchy of enrichment then rich investors would not be interested in AI at all. It's the only reason there's 22 million lying around to start training on a math problem on a whim; whereas the actual math researchers have to scrape together funding in hope of just maybe one day getting a 1 million dollar prize.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#807

Post by ModernMech »

chii wrote: post #739
ZYbCRq22HbJ2y7 wrote: post #645

Earlier quoted context omitted.

> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well. Because the training data is millions of hours human efforts being distilled into a cascading hierarchy of enrichment by interested parties without providing attribution or compensation?

> without providing attribution or compensation? many teachers also taught many students over the course of history, and very few would eventually pay any compensation or even attribute their financial (or career) outcomes to the teachers. What made model training different?

They attribute their success to their school and then donate to the school. Alumni donations are how universities stay afloat.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#808

Post by HarHarVeryFunny »

user43928 wrote: post #795
HarHarVeryFunny wrote: post #576

Earlier quoted context omitted.

Brubeck has admitted what he said, but claims he immediately retracted it as a "poor choice of words". Given Buckmaster's telling, this seems beyond "poor choice of words"... It was a veiled threat, that he then doubled down on with his "If you don’t want me to be nice, then I don’t have to be nice." follow-up. ** I said that if OpenAI released its result in the way proposed I would go public with what happened. The…

Can you specify what leverage you think Brubeck has over an independent professor's career in order to make threats? I see none, and consequently Brubeck's explanation makes more sense to me. I understand he meant these words, which he supposedly retracted on the spot, in a "why would you ruin your career with this behavior / turning down the opportunity I am offering" way. You don't think that is the likely explanat…

I'm not sure the value of speculating, but since you insist...

If someone said to you "you're going to regret this, mo-fo!", do you really need to suppose they had a specific plan in mind, rather than just intent to intimidate?

If you want to suppose that Brubeck had a concrete plan for how he would ruin Buskmaster's career, then my best guess would be that he was threatening for OpenAI to publish without giving any credit to Buckmaster (who has been working on Euler overall for at least 10 years). Obviously this would be absurd, but no more absurd than what Brubeck was also demanding - that Levant was excluded for any credit and could not have his name on any OpenAI writeup (with this being the exact sticking point that Buckmaster was refusing to accept).

The funny thing is that this threat, as is often the case, seems to say more about the insecurities of the person making the threat than the person they are directing it to.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#809

Post by ericmay »

Maybe these academics should learn to code now that they’re about to have their lunch eaten too. Boo hoo.

Power imbalances exist within academia as well. The professor at MIT has much more access and probably a lighter teaching load than a professor at, say, a big state university. Labs, teams that help you write grants, a shit load of money, book deals, podcasts, and much more.

If academics want to publish their work first they’ll have to compete and figure out how to do that. At the end of the day what matters for humanity is that the problem is solved, not whoever’s ego is stroked by being the first. In the spirit of collaboration shouldn’t they have been publicly sharing their work through every step? Maybe had they done that months or years ago some other researcher could have solved it even faster! Wait, what’s the point? Who solves it, how fast it is solved, or whether or not a human solved it? Times are a changin’. Get with the program. If any of the charlatans are to be believed this is the big one. It’s the steam engine on steroids. Now what?

The public (HN is a tiny bubble that doesn’t represent the population broadly) is looking around and saying wow, so some researchers thought they were close and OpenAI turned its attention to this problem and just went and solved it? Cool. They don’t care about some pissing match about vague ideas of stolen conversations when they are delivering Uber Eats to your dorm room.

With that said and now that I’ve anchored on an unpopular idea, I welcome my demise in the comments section. Woe to me and my karma :p

Re: More questions about whether researchers can trust OpenAI with unpublished math

#810

Post by mafuy »

munksbeer wrote: post #796
keeda wrote: post #632

Earlier quoted context omitted.

But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable. I suspect the main reason the community is not receiving it well is largely…

Hold on, you've just ignored the point of the post you're responding to. What isn't being received well is hearing that others are close to publishing on a solution to a problem, so quickly using your power imbalance (millions of USD and access to way better models) to front run this. Even if their model wasn't trained on the conversations, this is just a dick thing to do. That's it, that's why it isn't being receive…

It is simply unethical, period. Knowing that a solution exists is a gigantic advantage when working on a solution. Normally, noone can abuse the knowledge fast enough to gain an advantage, but here, they could. This is fraud and as a journal, I would reject it.
Post reply on HN