241–250 of 446 posts
Earlier quoted context omitted.
>I'd like to know a lot more about how that works. My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token. Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output wi…
> Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. Wouldn't you need the prompt to know the probability of the next token?
There are words/tokens that are heavily correlated to the prompt (a yes or a no, for example), and then there are others that are going to be less so (adjectives with a lot of synonyms for example).
Given a text, you can identify what the "load bearing" and auxiliary words/chunks are. Then, looking only at the auxiliary words/chunks, you should, in principle, be able to determine what other wordings could have gone there instead. From this, you can, very roughly, recreate the token probability distribution that was in effect when those tokens were generated. With the probability distribution in hand for enough chunks of text, you can start inferring properties about the RNG process that was used to sample from those distributions.
But then, this notion of "load bearing" vs "auxiliary" can be expressed directly in the probability distributions. A load bearing token just has a very high probability, and thus any RNG bias that may have been in effect will likely be swallowed in the distribution. So the parts of the text that are highly dependant on the prompt will naturally not be contributing much information about he RNG in the first place.
We need to just stop pretending we can reliably tell if plain text is written by an LLM. It’s just not a reasonable ask.
True, but what you can do is a one-sided guarantee. If it bears the mark, it is likely generated (or someone deliberately made it look generated). Thus, if a news article, research article, book, student paper submission, blog post , HN comment, etc, bears the mark, it could be automatically flagged as such. It helps detect low effort slop. --- Caveat. If you write your own creative work and send it to Claude for "cl…
Earlier quoted context omitted.
I am wondering how that applies to newly generated code. Odd variable naming? Stylistic choices that are watermarked? Or as someone else noted further down in the comments, it could be more subtle: Between the first and second most likely choice, in certain positions it will consistently choose in a certain way.
Personal observation: Opus 5, over the last week, has started outputting A LOT more comments. Despite my global instructions being full of variations on "don't use comments unless absolutely necessary". I might be imagining things of course. But comments would be great fit for this use case.
Earlier quoted context omitted.
Please don't let the arbitrary selection of phrase distract you from the substance of my argument: a product that I pay for is at best no better due to this change, and highly probably worse. Why am I paying for a tool that is beholden to clandestinely satisfy some far away master?
Good question! Why are you paying for some tool that has always been beholden to some faraway master's opaque agenda?
Earlier quoted context omitted.
I'm curious about your thoughts on pangram. I only really see posts on Reddit claiming it falsely labels their content as ai generated but nobody will actually post examples of "textbook from twenty years ago" or upload screenshots of a journal (also those posts usually feel deeply ai generated without an ai detector) Do you think this is an impossible task and we shouldn't try to solve it? Or do you think it's doabl…
This feels testable - you could go to fanfiction or similar sites with billions of words of writing from before 2016 or so and run them through it. I tried a chapter just now and got human doing that, but I'm not invested enough to run a hundred samples today. But it sounds like it would be an alright way to audit it? I will confess I'm pretty skeptical you could ever eliminate false positives here though. I can ofte…
Earlier quoted context omitted.
It's pretty trivial to command it to not speak that way. That's one of the first things you should write into the prompt. What style you want it to write in. Make it use a very concise and dry academic style with no overt LLMisms, melodramatic or flowery language, or metacommentary.
People have been posting some variant of this comment for three years, and it's no more true today. Ever notice that the "prompt engineer" career hasn't materialized?
Earlier quoted context omitted.
This feels testable - you could go to fanfiction or similar sites with billions of words of writing from before 2016 or so and run them through it. I tried a chapter just now and got human doing that, but I'm not invested enough to run a hundred samples today. But it sounds like it would be an alright way to audit it? I will confess I'm pretty skeptical you could ever eliminate false positives here though. I can ofte…
That still might work better with older texts. As AI-generated text gets more prevalent, I'm guessing people will start subconsciously adopting AI writing styles.
Earlier quoted context omitted.
>I'd like to know a lot more about how that works. My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token. Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output wi…
They still need to choose when to do that though. When I prompt the program to e.g. alter a bash script in a specific way or to recite a longer known text it can't go round and randomly exchange tokens. It has to somehow define what is a simple repeated text from a different origin and what is a novel generation.