Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

81–90 of 845 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#81
post #18

It’s crazy to me that companies/researchers share important data with these AI labs, you’re basically giving them your secret sauce which they then share with all of your competitors via training on conversations. At the same time I don’t really know alternatives other than a slightly less than frontier local LLM. Not sure how good they are at math.

Academic work is based on worldwide sharing, the sharing is not the problem, it's the lack of attribution. Unsurprisingly, these companies neglect standards of academic honor and attribution. Some human researchers also used to do that but in a discipline like mathematics this used to be a small problem because people tend to be so specialized that very few people could just grab someone's research and quickly piggyback on it, and if they do, colleagues will generally understand what happened. Unfortunately, AI is changing this.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#82
OpenAI is showing the world why they shouldn't trust AI hosted on some cloud somewhere.

If they're stealing math proofs to advertise their models, who's to say they won't steal your businesses IP to gain a competitive advantage?

They're not to be trusted with your data. I can't believe how short-sighted this is, they got a quick PR win at the expense of a much larger trust problem.

I wouldn't trust cloud AI at all at this point. Get an open Chinese model and host it yourself somewhere. The initial costs might be higher, but you'll break even pretty quickly and nobody will be able to steal your innovations.

This is American AI companies committing suicide.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#83

Earlier quoted context omitted.

"Discovery" is not a goal in itself. I could launch a project to find out how many people in the United States have names such that if you assign numbers to every character and then sum the values, the sum works out to 72. It's discovery, but it's useless unless it has some higher goal. The labs are attacking these problems as a demonstration of capabilities, spending more money on the demos than any mathematician wi…

Right, mathematicians care about clout and tenure, which is a much higher purpose.

I don't know about you, but if I apply myself fully to a problem and study it to the point where I'm literally one of the world's experts on it and then some assholes in Silicon Valley take my research and claim it for themselves, I will probably not feel too great about that...

Re: More questions about whether researchers can trust OpenAI with unpublished math

#85

Only after reading this post did I learn that my preferred AI trains on my inputs (prompts). How was I not aware of this before?

Everything. Your prompts, your conversation as a whole, public data, private data, usage metadata. It all goes into the big data machine.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#86
post #17

Earlier quoted context omitted.

The AI only seem to solve the problems that it had human trading data on… If this wasn’t human driven, I’d expect to see other problems within that problem. Space solved not just the ones that it had chat data on.

There have been about 6-8 major math breakthroughs claimed by AI. Only for 2 of them there are public accusations about the training data.

Only? That doesn't look small to me.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#87
post #9

Some mathematicians I know who've been following this have realized that they'd all gotten some emails from people they now know to be affiliated with OpenAI/Anthropic asking questions about their research in a way that seemed like scooping attempts. Also, a lot of my mathematicians buddies have reported students basically asking if it's worth ever doing grad school for pure math, and even very motivated students are…

You can't blame students for not seeing academia as the holy grail of knowledge anymore, when all the dialogue about technology and discovery has shifted to the hands of two private corporations

Re: More questions about whether researchers can trust OpenAI with unpublished math

#89
I must say, there is some weird feeling in knowing that great minds are naive enough to believe OpenAI wouldnt use their chats in any way. If you give a company information it will be used, regardless of laws or promises.

There is no prove in a world the AI companies would give to you ensuring that they didnt train or use the chats.

Why would you need to train a model on certain specific near prove chat if you just query it?

Besides that, its hard to believe that its the case for every "company stole my prove".

Re: More questions about whether researchers can trust OpenAI with unpublished math

#90
post #4

TL/DR: Mathematician opted out of training on 29-JUN and asked OpenAI whether they trained on his data and was told that it "did not happen" but it clearly did.

Well, OpenAI said "we didn't read the conversations", but they never discarded that the model was training with that data... so even worse.
Post reply on HN