Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Opus 4.7 knows the real Kelsey

theargumentmag.com

Post reply on HN
Print view View original

21–30 of 284 posts

Re: Opus 4.7 knows the real Kelsey

#21

Post by post-it »

_--__--__ wrote: post #7

On some level it would make sense for LLMs to be inherently good at stylometry, but apparently no model before Opus 4.7 could do this. And the one stylometric task that has been tried over and over with little reliability (here's some text, is this LLM generated?) is much simpler than identifying a specific blogger or a member of a small discord community. Not sure what to make of this.

> is much simpler than identifying a specific blogger or a member of a small discord community

Is it? I would think that identifying text written by a specific person is going to be significantly easier than identifying text distilled from the words of almost everyone alive.

Re: Opus 4.7 knows the real Kelsey

#22

Post by mikestorrent »

CTDOCodebases wrote: post #18

Maybe it’s time to start running a local model with a browser extension to defend against this type of stuff. Remember how the TrueCrypt project shut down shortly before a join goverment/university paper was released about code stylometry? I guess LLMs will be employed as a defence against that type of thing.

How does that defend against something having trained on a corpus of your own previous writing?

Re: Opus 4.7 knows the real Kelsey

#23

Post by tekacs »

A moderately well-known physicist and I talked about this a few years ago. He had been given access to the raw (non-instruct) version of GPT 4 as an early tester.

He explained that when he fed it snippets of the beginning of text, it would complete it in his voice and then sign it with his name.

I think this has been true for a while, probably diminished a little bit by the Instruct post training, and would presumably vary by degree as the size of the pretrain.

Re: Opus 4.7 knows the real Kelsey

#24

Post by afro88 »

jwpapi wrote: post #3

Could this be just memory? Not clear it actually isn’t

It's not, but the author did say they have used this test against models when they come out. So it's possible that put the unpublished text into the training data for the next model, somehow linked back to the author's identity

Re: Opus 4.7 knows the real Kelsey

#25

Post by nextaccountic »

tekacs wrote: post #23

A moderately well-known physicist and I talked about this a few years ago. He had been given access to the raw (non-instruct) version of GPT 4 as an early tester. He explained that when he fed it snippets of the beginning of text, it would complete it in his voice and then sign it with his name. I think this has been true for a while, probably diminished a little bit by the Instruct post training, and would presumabl…

> He explained that when he fed it snippets of the beginning of text, it would complete it in his voice and then sign it with his name.

Is this public text already in the training set, or private text that might as well be written on the spot for the AI?

I don't doubt AI can "fingerprint" you through your text (ideas, vocabulary, tone, etc), but those are different things, capability-wise

Re: Opus 4.7 knows the real Kelsey

#26

Post by superfrank »

alyxya wrote: post #15

I tried the four pieces of text with Opus 4.7 (in incognito) and it guessed correctly for two of them, and I made sure to specify no web search and the model seems to have obeyed my instructions with that. Although this is just a single piece of text from a prolific writer, it'll go much further with deanonymizing anyone when combining multiple pieces of text plus other contextual information about the writer that mi…

How widely known were the pieces of text? Are we talking about a section of MLK's I Have a Dream speech or hand written birthday cards from your grandma?

I'm using those as the two extremes, but if it's anything by anyone moderately well known (even a lesser known piece of writing), I'm not too surprised that it didn't need the web to figure it out. It's like if you showed me a Wes Anderson film or played me a Bob Dylan song I'd never seen/heard before, I could probably still figure out who it is without looking anything up. I don't think it's surprising that an LLM can do that much better than a human can.

Now, if you're giving it things like personal emails between you and your family and it's able to guess who you are, that's much, much scarier.

Re: Opus 4.7 knows the real Kelsey

#28

Post by Retr0id »

atleastoptimal wrote: post #2

One should assume that models will be good enough in the nearish future that privacy will be a thing of the past. Every anonymous post you made online can be traced back to you. However at that point AI will be good enough at fabrication that nobody will believe anything.

One "solution" would be to have an AI rewrite your posts into a neutral style (I hate the idea of this though...)

Re: Opus 4.7 knows the real Kelsey

#29

Post by H8crilA »

mikestorrent wrote: post #22
CTDOCodebases wrote: post #18

Maybe it’s time to start running a local model with a browser extension to defend against this type of stuff. Remember how the TrueCrypt project shut down shortly before a join goverment/university paper was released about code stylometry? I guess LLMs will be employed as a defence against that type of thing.

How does that defend against something having trained on a corpus of your own previous writing?

Exactly as much as closing your eyes and covering your ears.
Post reply on HN