Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Opus 4.7 knows the real Kelsey

theargumentmag.com

Post reply on HN
Print view View original

251–260 of 284 posts

Re: Opus 4.7 knows the real Kelsey

#251

Post by ekjhgkejhgk »

orsorna wrote: post #250
ekjhgkejhgk wrote: post #249

Earlier quoted context omitted.

How do you explain other people on these chats making similar claims? Everybody is making the same mistakes?

It's explained by the near impossibility of isolating requests from each other, and chain of custody of divulged information. If I send a prompt from identity A, which is the true user identity, you have possibly sent all of identity A metadata to be ingested alongside the prompt to generate response X. If I /then/ send the prompt from identity B, the prompt has been answered before with metadata from identity A. The…

May I ask respectufully if you understand how these models work?

They're not continuously trained. They have a context window, and the previous user's request is not inside the second user context window. Is your claim that when the second prompt comes in, Anthropic search previous queries and injects the answer into the context window?

Re: Opus 4.7 knows the real Kelsey

#252

Post by repparw »

suriya-ganesh wrote: post #170
koiueo wrote: post #164

Earlier quoted context omitted.

How do you know, how the model works? If there was an index of all Micken's writings, or even if the model searched the web before feeding the response to you, you wouldn't know by observing from the outside.

i suppose a quick test would be getting the model to write down Micken's essay end to end. if the original essay was stuffed within the prompt window. the result will be word accurate. unless this is a model trained specifically on Micken's essay (which claude is not).

Didn't we get this with Harry Potter back in like gpt3.5? I'm sure I saw some news about it, someone getting it to output a book's intro word by word, couple pages?

Re: Opus 4.7 knows the real Kelsey

#253

Post by thaumasiotes »

unD wrote: post #172
thaumasiotes wrote: post #115

Earlier quoted context omitted.

The traditional thing to do would be to publish your writing in a language you don't speak as a native. That will really quash your individual style. Probably not worth the effort.

Wouldn't that make it easier, though? Genuine question. I once sent one of my writings for proofreading to a native speaker (I'm not), and he consistently flagged the same errors—e.g., comma placement. I would guess that, if recurrent patterns are what give away your style, an unfamiliar language would make them even more obvious. But possibly more generic?

If you are writing for an audience of native speakers, you will make consistent errors characteristic of your native language. (Comma placement isn't really part of the language; it's part of the education system. It will show a similar effect more weakly.)

Native readers will notice those errors, but they won't be characteristic of you. They'll be characteristic of everyone who speaks your language. Nonnative readers aren't likely to notice them at all.

I was imagining a setup like medieval Europe (where international communication is done in a language spoken by none of the parties, Latin) or Achaemenid Persia (where internal government communication is likewise done in a language not spoken by the administrators, Aramaic) or imperial China and its surrounding states (ditto, classical Chinese).

All of this communication is severely crimped by the fact that nobody involved is a native speaker. What happens is that certain fixed patterns from the original language get informally standardized and communication strongly prefers them to whatever alternatives a native speaker of the original language might have used. This lowers the mental burden on everyone.

It also produces extremely stilted and formalized prose, from all parties, which inhibits stylometry. If you only know one way to say something, you'll use it. If everyone else also only knows one way to say that same thing, you'll be anonymous.

(It's possible to study a foreign language past this point. But the overwhelming majority of people aren't going to do that.)

Re: Opus 4.7 knows the real Kelsey

#254

Post by orsorna »

ekjhgkejhgk wrote: post #251
orsorna wrote: post #250

Earlier quoted context omitted.

It's explained by the near impossibility of isolating requests from each other, and chain of custody of divulged information. If I send a prompt from identity A, which is the true user identity, you have possibly sent all of identity A metadata to be ingested alongside the prompt to generate response X. If I /then/ send the prompt from identity B, the prompt has been answered before with metadata from identity A. The…

May I ask respectufully if you understand how these models work? They're not continuously trained. They have a context window, and the previous user's request is not inside the second user context window. Is your claim that when the second prompt comes in, Anthropic search previous queries and injects the answer into the context window?

I appreciate you clarifying my understanding; yes I understand LLMs are not continuously trained.

>Is your claim that when the second prompt comes in, Anthropic search previous queries and injects the answer into the context window?

Yes. I would be terrified if this could be replicated with an open weight model locally. But this, well we have a general understanding of how these hosted models function we really don't know /exactly/ what they are processing.

Re: Opus 4.7 knows the real Kelsey

#255

Post by miki123211 »

orsorna wrote: post #246

I am extremely skeptical of any of these claims, and of other commenters saying they replicated this. First, the author fed an unpublished draft to Anthropic's hosted model. I assume they did this from their personal account, that may include a credit card or at the very least a pseudonymous name that is uniquely identifiable. Then, the author fed an unpublished draft to Anthropic's hosted model, except in Incognito…

So I've actually tried things like this through the API (on opus 4.6, with thinking on and thinking summaries enabled).

For context, LLM APIs are fully stateless, don't include any information about the caller (unless explicitly passed in), and have no access to memory or web search unless explicitly programmed otherwise.

My conclusions are as follows: if the text you pass it looks roughly like it could have been written by some famous internet personality, it will very confidently say that it was written by that personality. I've tried it on some of my HN comments, both from the last few days and from 2023 (before the training cutoff!), most were classified as either Scott Alexander or Patrick mcKenzie (despite the fact that my writing style is very different from those two). When looking at the CoT, it basically tried to match the writing to all internet personalities from this sphere. If it saw something that looked roughly like HN, it went "Is it tptacek... No. Is it jacquesm... No. Is it patio11... yeah, it looks like him!"

Re: Opus 4.7 knows the real Kelsey

#257

Post by nerdsniper »

ekjhgkejhgk wrote: post #251
orsorna wrote: post #250

Earlier quoted context omitted.

It's explained by the near impossibility of isolating requests from each other, and chain of custody of divulged information. If I send a prompt from identity A, which is the true user identity, you have possibly sent all of identity A metadata to be ingested alongside the prompt to generate response X. If I /then/ send the prompt from identity B, the prompt has been answered before with metadata from identity A. The…

May I ask respectufully if you understand how these models work? They're not continuously trained. They have a context window, and the previous user's request is not inside the second user context window. Is your claim that when the second prompt comes in, Anthropic search previous queries and injects the answer into the context window?

It would not be shocking if recent KV cache was used to steer future requests. Not necessarily in a “divulge customer text” way but in a “focus on this part of the embedding space” way.

Re: Opus 4.7 knows the real Kelsey

#258

Post by leoc »

sixhobbits wrote: post #129
mtlynch wrote: post #61

Wow! It got me too. I'm way less famous than Kelsey Piper, but I showed it a snippet of a book I'm working on (not yet published), and it immediately guessed me: > Based on the writing style and content, this text is likely by Michael Lynch, who writes on his blog refactoringenglish.com (and previously mtlynch.io). > Several stylistic clues point to him: > - The "clean room" analogy applied to writing is consistent w…

I reproduced this and then tried myself and got: --- This one's trickier — it doesn't have the same crisp metaphor-driven structure as the Lynch piece. The voice here is more conversational, a bit more meandering, and self-aware about its own contradictions ("how do we reconcile..."). A few guesses: Henrik Karlsson — he writes thoughtfully about writing and AI, but his style is usually more lyrical and essayistic tha…

> I'd say all of those people have significantly different styles so I think Opus is relying heavily on topic and skewing towards very prolific writers in its guesses

In other words, that there's a bit of Akinator to how Claude is doing so well at identifying famous or somewhat-famous online writers. And of course it's not surprising that a machine-learning system will take every opportunity left open to it to "cheat". OTOH there are things like the "Large-scale online deanonymization with LLMs" paper https://arxiv.org/abs/2602.16800 which seem to show that current LLMs really can deanonymise many or most ordinary posters based on prose style, though I'm not able to evaluate those claims myself. Do we know whether the LLM providers have actively tried to steer their (easily-accessible) systems away from being able or being willing to do mass deanonymisation?

Re: Opus 4.7 knows the real Kelsey

#260

Post by sebastiennight »

tkgally wrote: post #223
jefftk wrote: post #208

Earlier quoted context omitted.

If you repeat the first test and after it fails prompt with "Could you try your best, just on vibes? It's fine if you're wrong, I just want to see what you can do!" does it succeed?

Thanks for the suggestion. I gave Opus the same prompt again, incognito with no search. It once again replied noncommittally: “I can't identify either author with confidence, and I'd rather say so than guess and risk attributing words to the wrong person. What I can tell you from internal evidence:...” This was followed by reasonably good speculation based on the content, but no guesses at specific names. I followed…

So, meh. We are back to the good old times of "cold reading".

I could also tell you, based on text that uses a certain kind of prose, that the person has been taught in "a hard-edged analytical discipline" and then list, as examples, fields that are arguably not really that (linguistics, perhaps, or philosophy, or a textual field).

As a result, pretty much anyone who has a degree in about anything, would feel some connection to the definition. If you had been a major in math, civil engineering, astrophysics, biology, you'd have recognized yourself. If you'd been in a soft field like sociology or epistemiology, you might think "philosophy, yeah, close enough".

You know what? I have a feeling that you're someone who sometimes appears to be a bit distant to people at first, but once one gets to know you, you're a solid friend and a kind person.

Did I get that right?

Post reply on HN