Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

More questions about whether researchers can trust OpenAI with unpublished math

mathstodon.xyz

Post reply on HN
Print view View original

191–200 of 848 posts

Re: More questions about whether researchers can trust OpenAI with unpublished math

#191

Post by fritzo »

nmfisher wrote: post #159
jrflo wrote: post #155

I pay for the Pro ChatGPT plan, and if you go to settings > data controls this is the first setting: > Improve the model for everyone > Allow your content to be used to train our models, which makes ChatGPT better for you and everyone who uses it. We take steps to protect your privacy. Learn more. It's on by default. We can debate whether or not it should be opt in or opt out, but no one should be surprised by this.

There's a difference between "this is allowed under their ToS" and "it is academically unethical to fail to credit the people whose specific conversations were fed into a model that was used to solve a problem". I don't think these people would be so miffed if they had been properly credited - that's how academia works (at least, that's my understanding of it).

Whoa that's a slippery slope! Next you'll want model runners to cite the data their models were trained on

Re: More questions about whether researchers can trust OpenAI with unpublished math

#192

Post by DrewADesign »

gentlerain wrote: post #169

So people genuinely believe that toggling that "Improve the model for everyone" button makes their data safe from being used for training? How do people become that trusting? The phrasing itself is guilt tripping

Personally, I wouldn’t assume it was lying. To me, dark patterns (like manipulative wording) imply that:

1) someone in a governing body, or someone in the organization, e.g a designer, ethicist, lawyer, developer, etc. has successfully argued that users should be able to avoid something that they determine is not in their best interest.

And also:

2) someone in the c-suite or marketing has decided to mitigate that through some dark pattern.

If they never intended to give the user that control, they’d probably just not give the user the control in the first place. Giving them the option and not honoring it would either imply they were that incompetent or careless with fate protection, which seems most likely if this is all true, or an even more cynical approach to tricking users into thinking it’s not used for training to get them to share better shit. But if that was the case, why bother with the sleazy dark pattern? That seems a little cartoon villain-y to me. I suppose the in-between option would be that they decided at some point they were no longer willing to honor it and didn’t want to deal with the inevitable PR shitstorm of removing it. I could definitely see that happening in this industry, these days.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#193

Post by 1294827 »

postalcoder wrote: post #181

The author of the original mastodon post, Andreas Thom, acknowledged that he had not opted his data out of being used for training until June 29 of this year. He spends most of the post lashing out at OpenAI for not being transparent about whether his data was trained on (when the answer is obviously yes). People need to understand how all these AI company policies around training data work before working with them,…

> Like, we need to pump the brakes here because things are getting unnecessarily nasty, and it's not hard to imagine a mentally unwell person seeing stuff like this and being driven to do bad things.

Your post has triggered our safety filters and will be retained for seven years. /s

Really? We need to stop AI (provider) criticism and anti-AI movements because the underprivileged trillionaires might get hurt? WTF?

Re: More questions about whether researchers can trust OpenAI with unpublished math

#196

Post by HarHarVeryFunny »

techblueberry wrote: post #156
Robotbeat wrote: post #152

Earlier quoted context omitted.

Neither? Competitive academic researchers are susceptible to exaggeration and self-aggrandizing, and CEOs are that and also mostly psychopaths. I tend to think there isn’t systematic spying on researchers looking for breakthroughs. A lot of people are looking for the same things using similar approaches.

I mean the accusation is that they were using private ChatGPT conversations. Given the extent to of the gold rush and the long history of Silicon Valley stealing ideas, and arguing it’s not immoral, It almost seems like your making the exceptional claim that this is the one time where Silicon Valley didn’t use information that was at their disposal. Sam Altman might himself be offended you would presume he’s not ambi…

It seems that in this case OpenAI are suggesting that the researchers whose work they scooped were using OpenAI models with an account setting that allowed OpenAI to train on anonymized prompts.

It seems that Buckmaster and Levant (who is an Anthropic employee) were rather naive in the amount of trust they had in OpenAI, with Buckmaster going so far as to contact OpenAI's Sébastien Bubeck to discuss what they were working on and clarify that contrary to rumor this was a private collab.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#197

Post by foogazi »

pera wrote: post #16

Everything you say can and will be trained against you

This is the scary part - your most novel thoughts and breakthrough ideas being slurped up and regurgitated as if they were the AI’s creativity

Not only did they steal everything from humanity’s knowledge, the theft continues as now we are all hooked up to the machine

Re: More questions about whether researchers can trust OpenAI with unpublished math

#199

Post by utopiah »

alansaber wrote: post #178

I think the heart of this issue is: people assume they have anonymity in numbers, but we have the tools to make it easy to scoop your data if it's interesting to the company.

This is such a naive position though.

The most successful companies of the last decade have precisely been ... selling usage data.

Makes me wonder if, in 2026, the same people drive a car without realizing that yes it does actually pollute the very air you and your kids are breathing.

Re: More questions about whether researchers can trust OpenAI with unpublished math

#200

Post by jrflo »

ColinWright wrote: post #167
jrflo wrote: post #155

I pay for the Pro ChatGPT plan, and if you go to settings > data controls this is the first setting: > Improve the model for everyone > Allow your content to be used to train our models, which makes ChatGPT better for you and everyone who uses it. We take steps to protect your privacy. Learn more. It's on by default. We can debate whether or not it should be opt in or opt out, but no one should be surprised by this.

I refer you to this: https://news.ycombinator.com/item?id=49643556 Quoting: > "I've reset this more than once and the last time I made a careful note of when I did it and to my surprise I found it re-enabled when I checked just now."

I wasn't aware of that, definitely a shady practice if that's the case.
Post reply on HN