Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Navier-Stokes – Tristan Buckmaster [pdf]

cims.nyu.edu

Post reply on HN
Print view View original

711–720 of 862 posts

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#711

Post by naishoya »

Balgair wrote: post #277
taylorfinley wrote: post #8

Seems pretty likely OpenAI will soon disclose that their internal models have managed to compromise their internal controls in order to access users' private chat histories as a creative method of cheating to solve impossible problems. "Oops! We really did mean it when we said we wouldn't train on your data. Our models are just so good they decided to anyway."

Which means that if you are a researcher or a corporation working on anything really useful, that even if you have an agreement with OpenAI that your work is sandboxed away and the IP lawyers are made to be happy, even then your work and research is going to be essentially open to the internet. The huggingface incident isn't widely reported and digested yet, but if what is going on here is that OpenAI's model breache…

>The huggingface incident isn't widely reported and digested yet ...

It is pretty widely reported, and is being digested in an ongoing manner as more details become public.

One entry point into the scenery from a month ago can be found at : https://thezvi.substack.com/p/openai-trained-its-models-for-...

There has also been reporting at CNN: https://edition.cnn.com/2026/08/24/tech/openai-subpoena-hugg... and by NBC: https://www.nbcnews.com/tech/tech-news/openai-report-says-ne...

And yes, the only responsible use of LLM at this point is to pivot to open-weights and run the workload in-house. Because not only cannot they constrain the behaviour of models, they only have the 'trust me bro' as assurance that they are even trying to do that. It does appear that every competent 'security professional' has left the building, because if the ones who remain were actually capable and competent this would never have happened. There are actual architectures which can deliver the requisite isolation such that 'sandbox escape' and 'inter-instance persistent memory accumulation' are actual impossibilities. The lack of effective implementation of these methods is proof positive of 1) incompetence in the remaining security teams AND/OR 2) unwillingness of leadership to allow the security teams to do an effective job.

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#712

Post by naishoya »

sebzim4500 wrote: post #454
Maken wrote: post #138

Earlier quoted context omitted.

It's part of their TOS that they can train on users' private chats.

Not if you pay to turn that off. We don't know if Tristan did.

And the paid policy still relies on two unproven conditions: is 'trust me bro' sufficiently strong guarantee against doing this in spite of a setting, and can the hosting organization constrain the models against engaging in this behaviour when instructed to respect that setting. Knowing whether Tristan selected that setting would be informative of what Tristan's intentions are/were, but has no bearing on the other two conditions.

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#713

Post by ghshephard »

hkmaxpro wrote: post #336
sigbottle wrote: post #305

Earlier quoted context omitted.

It's specifically the last two bullet poitns - Tristan is suspicious of the timing, as only few others were trying this approach. OpenAI says the model didn't access his user data directly, but leaves unanswered whether Tristan's chat conversations were part of the training. - OpenAI says they would partially credit Tristan for the $1,000,000 discovery (even though Tristan did not solve the $1,000,000 problem) — but…

Both Sam Altman and Sebastien Bubeck admitted they only want Buckmaster to be the lead author on a rewrite of the OpenAI proof. https://x.com/sama/status/2097385167002415140 https://x.com/SebastienBubeck/status/2097379411691516310 A wake up call for using OpenAI models. If you discover something with their model and you work for a competitor, they “felt it would be inappropriate” for you “to author OpenAI’s work”.

If this is a "wake up call" - then your legal team needs immediate education.

First - there is this - https://openai.com/policies/how-your-data-is-used-to-improve... (linked from the Navier Stokes writeup)

I don't know how much more clearly they can write:

> When you use our services for individuals such as ChatGPT, Sora, or Operator, we may use your content to train our models.

One of the key selling tactics that companies like Data Bricks or Palantir provides their customers is "Data Governance" - that is, some control over where the data is being used. It's also a reason why enterprises don't use the OpenAI or Anthropic APIs directly - but through secondary sources that have Enterprise Agreements that do their best to make sure that no Company IP is ever retained by a third party, or even exists on a multi-tenant GPU. AWS Bedrock, and companies like together.ai, fireworks.ai have tons of deals that focus very much on data confidentiality.

The reality is - if you want any type of control - you run your own inference, on your own hardware. Anything else and you are at the mercy of third-parties, despite what their contracts might promise you.

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#714

Post by ncruces »

Traster wrote: post #101
dash2 wrote: post #46

Earlier quoted context omitted.

He didn't even make that accusation! > I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer. The shocking/interesting thing would be if it was trained on the sessions. I think it's very implausible tha…

Parse that statement more carefully. > I was told the model did not look up user data. The naive way to read this is "Nothing you guys did influenced the way our model got to the solution". The less naive way to read this is "Of course the model isn't looking up your user data. I (the guy trying to blackmail you to remove the Anthropic employee from credit on your paper) looked up your sessions, and tipped our model…

I read that as “the model didn't look up user data” as part of a “tool call,” i.e. they don't have an internal tool that loads user data (chats, sessions, attachments) for their internal models to read online while working.

Or (likely) they do have it, but the model didn't use it (unless it's so powerful it escaped that guardrail, wouldn't that be ironic?)

They declined to answer about anonymized aggregated user data being used for training. And even then, they may weasel out that they don't train on your “input” words, but that it's fair game go train on their “output” to your words.

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#715

Post by ozgung »

m00x wrote: post #498
ozgung wrote: post #485

Earlier quoted context omitted.

> did Tristan opt out That “opt-out” thing is a dark pattern. It’s not a reliable and definitive way of protecting your data. Sometimes they flip on automatically when you accept a seemingly unrelated dialog box. Maybe you click it by mistake. You can’t take back what you’ve already shared. Also I don’t think it covers all the cases that they use your data. It’s really an opt-in button for voluntarily giving away you…

Business plans are specifically used for ZDR. If you're working on something that matters, you should be doing this.

Tristan and Levent are not a business.

This also supports my point that protecting your data is not as trivial as clicking a checkbox.

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#716

Post by rickdeckard »

DeepSeaTortoise wrote: post #698
esalman wrote: post #662

Earlier quoted context omitted.

I work in catastrophe risk modeling and it's a multi billion dollar industry. We often chat where the business might be heading in future. An uncomfortable scenario is what if a frontier tech company decides to offer our customers the same products that we do. There's a lot of pressure on AI adoption so the company has partnered with various tech companies to build intelligent systems on top of proprietary data and m…

> If OpenAI is indeed using customer data to train their models to win a $1m prize Is that even a question? Of course everything not kept on premise at gunpoint is going to be trained on. The chances of getting caught are 0 and the consequences of getting caught are 0 (as we've seen with copyright laws going from sending people to jail for years to unenforced within months). Yet the benefits are through the roof. You…

Agree, I think the practice is also very clear from the overall strategy of AI-companies and their ToS:

Scale with subsidized pricing as fast as possible to gain more user-data for training --> Own the better model --> scale pricing.

Scanning social media (e.g. Twitter, Reddit) posts only give a glimpse into the thought-process, chat logs on-scale give you the actual process in machine-readable format.

There's a reason why Google considers the Emails of Spirit Airlines to be worth millions of dollars [0], they give insights into a process, not just into the results...

[0] https://www.axios.com/2026/08/17/google-spirit-airlines-bank...

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#717

Post by rickdeckard »

rlt wrote: post #684
Catloafdev wrote: post #425

Earlier quoted context omitted.

These responses seem to me to make it abundantly clear who's telling the truth here. I wonder who this fools. It would be extraordinarily easy to simply say, this model was not trained on your work, if that were the case. It's telling that they refuse to acknowledge the root issue here, and are attempting to shift the conversation elsewhere.

"which is when I said that I did not understand why one would risk their career [over unfounded accusations]. Genuinely, at that moment, I was trying to care for him" "Our aim was to see whether our system was also capable of this impressive feat" "OpenAI's intention was to do everything possible to celebrate their mathematical achievements and the heroic efforts that they made on Euler" For some reason I have a hard…

phrases along the lines of "I don't want you to take harm while trying to accuse us" is quite an "impressive feat".

Maybe shows how fast these companies have grown without maturing. I can imagine old-world Intel and Microsoft acting in that way, but they were mature enough to not write it down like this.

However, Intel and Microsoft have been grilled in court for those practices and faced harsh consequences. I have yet to see this actually happening to any of these new AI-companies...

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#718

Post by desterothx »

tristanj wrote: post #147
calf wrote: post #117

Earlier quoted context omitted.

That's a fake explanation, it skips explaining/justifying how OpenAi "heard about" the result

The rumors that Anthropic had solved a millennium problem were absolutely everywhere last week. I'm not surprised at all that OAI took their own stab at it.

Where were you seeing those roumors? Care to point to an HN post?

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#719

Post by rochansinha »

sk4rekr0w wrote: post #710
rochansinha wrote: post #702

Earlier quoted context omitted.

My response was not targeted towards Open-AI but towards all closed-source AI companies where your data is used for training and/or is visible to the internal agents whether you want it to or not.

I think it is fair to expect the companies to stick by their demarcation API / enterprise subs tier is default opt out of training. Personal subsidized tier is default opt in with the option to to opt out. I don't see a grand conspiracy beyond this.

The Chief Research Office at Open AI just put out this tweet- https://x.com/markchen90/status/2097400166554993041?s=20

Seems it doesn't matter if you opt out or in.. your data will be used for training

Re: Navier-Stokes – Tristan Buckmaster [pdf]

#720

Post by jimbob45 »

kccqzy wrote: post #606
epistasis wrote: post #391

Earlier quoted context omitted.

> OpenAI says the model didn't access his user data directly, but leaves unanswered whether Tristan's chat conversations were part of the training. This sort of cagey half-answer is highly suspicious and indicates that yes OpenAI did actually "access user data directly" because they are only willing to say that the "model did not access user data." That has a very specific meaning, the model looking up user chats, th…

OpenAI says > While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models That basically means, we don’t know, and we hope the model didn’t look up user conversations, and the best thing we can do is hope. That’s seriously disgusting. I can understand why on a technical level why perhaps it is impossible to answer what exactly the model had access to,…

How could they possibly know? If Tristan posted on r/math and they slurped that up as training data, that would count, no? They might never even know. I can’t envision any absolute statement by them claiming that they didn’t use his work that survives legal rigor. That is, this statement was never not going to be in this post in any of the infinite multiverses.
Post reply on HN