I do wish we have a better way of doing this - the workaround given in the article just shows how fallible this is.
Proving a human wrote something
31–40 of 88 posts
Re: Proving a human wrote something
#32I mean the ability to fake all these metrics is relatively trivial, since I have written anti-bot fingerprinting scripts for automating various online services that want to keep you from automating them, you have the text you want to send from source A, you have random typing speed array you want to type them in, your chance of mistakes (put wrong character, backspace to remove, put in correct character). And this do…
on the other hand - my automation was based on not wanting to spam services but provide the minimum level of content posting and regularity to benefit from algorithms that boost content based on the poster's engagement level, automation that wants to spam cannot benefit from this because slowing things down to show as if it was made in real time by a human (with fully non-headless browser etc.) does somewhat defeat the ability to spam like a machine.
Re: Proving a human wrote something
#33I’ve been playing around with some similar ideas in a couple of pet projects. I truly believe that some type of “proof of work” system like this will be the only reliable way to have confidence in human writing. The various AI “detector” software that’s out there seems to be a dead end to me.
I can’t conceive of any system that couldn’t be gamed. The better bet, in my opinion, is gauging the quality of the writing, as we already do.
Re: Proving a human wrote something
#34I’ve been playing around with some similar ideas in a couple of pet projects. I truly believe that some type of “proof of work” system like this will be the only reliable way to have confidence in human writing. The various AI “detector” software that’s out there seems to be a dead end to me.
What exactly do you think is wrong with Pangram?
Re: Proving a human wrote something
#35I'm actively reviewing my writing and making it worse to avoid being accused of LLM slop. It's so difficult to write, you review, edit, edit and you read it and you can't unsee the LLM criticism.
I'm not doing that. I'm not going to be coerced into a writing style that's not mine by LLMs. AI detectors think that 30% of the stuff I wrote 20 years ago is AI-generated. If someone wants to think, incorrectly, that my writing was done by an LLM, I can't stop them.
Re: Proving a human wrote something
#36Detecting Ai use based on "how many keystrokes did you type and when" doesn't solve the problem, because someone can just write a program to mimic human typing in the words.
So the issue isn't "did a human write something", it's what the actual content is
Re: Proving a human wrote something
#37Just like we trained on human language to create LLMs, we can train on human keystrokes with a similar algorithm and spit out believable (at least statistically) "human" keystrokes generated by machine.
Yeah I think the "how can you prove text was handwritten" question is a subset of the larger "how can you prove that a computer is being driven by a human" problem that all of the work around captcha, attestations, biometrics, and government-id auth has been aimed at. The fundamental issue, it seems to me, is that any signal that a human can provide to a computer (keystroke, camera frame, mouse click, etc) is inherently only parsable by code because a sensor has translated the analog signal into a digital one. That same requirement also ensures that the input can be digitally spoofed or automated. There's a similar problem on the output side: how can an analog user trust a digital certificate? What's stopping me from copying the certificate HTML or taking a screenshot and using it to trick people into thinking my AI content is handwritten?
I don't have any suggestions. I worry that the only strong solutions require a lot of power to be given to a centralized authority.
Re: Proving a human wrote something
#38i do actually think the general thrust of the idea is good. i also think it's sort of inherently cursed, and has some amount of a predeterminable fate.
if it doesnt take off, it dies.
if it does take off and becomes a relevant currency of some sort, it will need improving. if it needs improving how far are we willing to go? does some alternative system fork off to handle severity of provenance concern?
what happens when AI action becomes indistinguishable from human action? what happens when sticking computers in your head becomes vogue?
if it takes off and fills a small niche, maybe thats the best future.
Like I said, cool idea, but seemingly very cursed from the get go.
Re: Proving a human wrote something
#39i do actually think the general thrust of the idea is good. i also think it's sort of inherently cursed, and has some amount of a predeterminable fate. if it doesnt take off, it dies. if it does take off and becomes a relevant currency of some sort, it will need improving. if it needs improving how far are we willing to go? does some alternative system fork off to handle severity of provenance concern? what happens w…
This is an arms race, advantage attacker.
That means this basically works, as long as it never gets large enough to attack. Which may suit the author just fine. Not everything has to solve the world's problems. But it won't generalize very far, no.
Re: Proving a human wrote something
#40> how do you prove a human wrote something? Forget about the why or the value in it, just: how?
> Semoi is a plugin (currently only available for Obsidian) which tracks the length of time it took for a document to be written up
Trying to mechanistically prove that a human created some content as opposed to ai, in the age of LLMs and style transfer when you can just ask for something to be written in the style of Mark Twain or drawn in a style of van Gogh and get a great output, is a fool's errand.
All solutions to this end are going to be some form of attestation.
Even the proposed approach of tracking keystrokes and timing as a form of mechanical attestation, is going to be short lived because someone will train an AI on a corpus of human keystrokes and get a replication. May not even need an ai for this, a stochastic program could conceivably reproduce this behavior.