Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stealing Reasoning Traces from Proprietary LLM APIs

stolen-thoughts.com

Post reply on HN
Print view View original

101–110 of 325 posts

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#101

Post by rfoo »

desterothx wrote: post #60
NitpickLawyer wrote: post #25

Earlier quoted context omitted.

> But the real question is: Is it okay to steal By definition it cannot be stealing since you're paying for the tokens. It may be against their ToS, depending on what you end up doing with those tokens, but it cannot be stealing. If they charge by the token, all your tokens are belong to you :) I also find it very strange that everyone sort of accepts their ToS like no big deal. Imagine MS using the same terms for th…

I love how some of the biggest advancements in llms came from the Chinese labs, yet people still jump to distillation being unreasonably effective. Distillation is very good at creating smaller models from large ones sure, but nothing to me indicates it is 'unreasonably effective' compared to all the other bells and whistles being iterated on

Let's face it. Chinese labs made some of the biggest advancements. AND training on Claude (or GPT) output IS unreasonably effective. The two sentences are true at the same time.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#103

Post by crabette »

supriyo-biswas wrote: post #79
myworkaccount2 wrote: post #27

Earlier quoted context omitted.

There seems to be an obvious choice to make here, should you give the users to decrypt and use the COT that they did not generate themselves? This is only required if you want users to be able to share things with everyone and you are going for the simplest implementation. If not you could try to keep a record of keys associated with a user, then when a new request comes in look through to see if the user has a valid…

It’s about being able to change models mid-task. For example, I want to be able to plan using Fable but implement the plan using Sonnet, and that won’t work if this is implemented.

For plan it's relatively easy, just make the plan the artifact. The point is to ingest knowledge with one model and use it in another, and that is not necessarily easily expressible in natural language.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#104

Post by Perseids »

niemandhier wrote: post #59

You cannot steal what is not owned. At least in the EU there is no copyright for LLM outputs, so I guess all they might do is violate the terms of service.

I think they are doing us a disservice by perpetuating this focus on intellectual "property" [1] in regard to AI.

The real issue with Anthropic, OpenAI etc. is not that they have used all of our public knowledge for training their LLMs. Creating new work from old and learning from prior generations is what we all do. The issue is that they want to claim all of the benefits for themselves. They are standing on the shoulders of giants and have contributed an inch themselves, yet want to privatize the power of the whole giant. We shouldn't let them "own" these models.

The influence on society by AI is so novel that it's reasonable to craft new laws specifically for them. There are a lot of ways to deal with their power grab. We could force them to open source the models after two years. Or we could tax tokens or compute. We just need to agree that the power grab is the problem, the privatization of our cumulative knowledge, and not some details about copyright infringement.

[1] I know I'm going to risk dissent just by putting quotation marks here. But I think for this topic specifically it is crucial to understand that intellectual property is an arbitrary social/legal construct. With physical stuff, there is an inherent scarcity. If you steal my smartphone, I no longer have it. If you steal the character from my book, I... have a harder time selling my next book? Our ancestors have invented copyright to solve a specific problem, but the solution has become perverted over time. There are a lot of egregious cases out there (looking at you, Disney), but even relatively tame success cases don't look good. Society has paid J.K. Rowling a literal billion for her work and still this cultural touchstone of a generation remains privatized. Imagine what other authors could have build upon her stories, if only they were allowed to publish their own stories with these characters. She has not been a particularly good steward in the past decades.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#105

Post by yubblegum »

dannyw wrote: post #83
yubblegum wrote: post #30

Earlier quoted context omitted.

Seriously, what does it take to encrypt per session? There are many ways to make it scalable and efficient so I am wondering if this is left like this to allow interested 3rd parties ahem unobtrusively peek what people are doing with the AI. (Thanks for the link. That’s an interesting idea!)

The provider has the hidden text anyway; this isn’t customer managed encryption.

Sure, but if each session has a unique key then these need to be managed and stored and unauthorized access to these leaves tracks. So all that had to be 'compromised' is a single universally applicable key. Again, the question stands: session based encryption can be scalable and efficient. Why aren't they using it?

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#106

Post by 8note »

Terr_ wrote: post #88
niemandhier wrote: post #59

You cannot steal what is not owned. At least in the EU there is no copyright for LLM outputs, so I guess all they might do is violate the terms of service.

While I feel it is morally OK for users to access the metadata of their own exchanges, there's something awry with that formulation, (not-)"stealing" is about more than just copyright or ToS. For example, consider my browser cookies that authenticate me to HN right now. Nobody even wants to copyright them, but if you were to somehow acquire a copy I'd very much consider it "stealing."

Im not so worried about that. Im worried about somebody impersonating me or doing something bad and not authorized with the cookies

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#107

Post by 8note »

tanh wrote: post #42

So to make the APIs stateless (the "ideal" where they don't use server side sessions/etc) we ended up with this. I'm sorry but this is kind of hilarious. Given the salaries paid to the workers at these companies and the hype of the models, I can't believe they all fell to the same flaw.

this is a lethal trifecta, but where a chunk isn't even needed

you have a secret to keep that is read by the llm, and untrusted input that wants to exfiltrate it.

by hell or high water, the agent is gonna output that text

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#108

Post by Barbing »

the_af wrote: post #98
Barbing wrote: post #73

Earlier quoted context omitted.

This is a non-transparent aspect of submitting a link to HN that is quite misleading. You think that you are adding a description for your post when in fact you're simply submitting a regular comment not promoted or distinguished in any way. It's even worse considering posts without URLs would take the same text from the same input box on the HN submission form and append it under the post title, locking it to the to…

Wow! I totally missed that the person I was replying to was the one who had submitted the link. What you describe is surely what happened. I feel bad now :(

There must be a reason HN does not colorize the OP username or something. But there totally could be some indicator of “post submission text” without too much in the way of negative consequences… (the fact this has never been added tells me I’m being naïve)

Feedback emailed to HN!

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#110

Post by cavisne »

Groxx wrote: post #4

> We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, ... Ha! I've been wondering if replaying across models would work, ever since https://blog.cryptographyengineering.com/2026/05/29/fooling-... I'm honestly rather curious if this was intentionally allowed, it's the sort of validation that's easy to miss (particularly if you're wading into the vibe waters). Seem…

You can switch models mid conversation so it makes sense that it is allowed. Otherwise you would lose all the reasoning traces.
Post reply on HN