Live data from Hacker News

Proving a human wrote something

gjtorikian.online

81–88 of 88 posts

Re: Proving a human wrote something

#81

Earlier quoted context omitted.

Right. But you'll never have a checker for that. How do you check for the overall idea being AI-shaped? It's a much deeper check than style, even typing patterns or anything like that. If I consult the chatbot, then memorize the overall arguments and ideas, and then record myself typing out an essay regurgitating it in my own words, how can you distinguish that from writing down my own ideas?

Fundamentally, I don't actually care about whether AI wrote it or not. I care about whether the ideas are well-thought-out or half-baked. I care whether the writing is slop (whether AI or corporate PR or politician-speak or whatever), or whether the writing is actually something a human would want to read. But as you say, we don't know how to write a checker for that...

I fully agree with you that whether AI wrote it or not isn't the actual issue.

Luckily, it is fundamentally easier to measure the quality of the writing and argument than it is to determine whether a human wrote a text or not in the face of motivated "attackers" with access to whichever judging mechanism you use.

The problem will be for the people who do care whether AI wrote something or not rather than about the quality.

Re: Proving a human wrote something

#82

Earlier quoted context omitted.

Except quite oddly, virtually no one has collected precise keystroke information or if they have it certainly is not public. I genuinely wonder if such a dataset exists

Online typing speed tests are in a really good position to collect and sell this data, if they aren't already doing it.

Typing speed tests are not likely to end up reflecting typical typing patterns, though.

Re: Proving a human wrote something

#83

I mean the ability to fake all these metrics is relatively trivial, since I have written anti-bot fingerprinting scripts for automating various online services that want to keep you from automating them, you have the text you want to send from source A, you have random typing speed array you want to type them in, your chance of mistakes (put wrong character, backspace to remove, put in correct character). And this do…

on the other hand - my automation was based on not wanting to spam services but provide the minimum level of content posting and regularity to benefit from algorithms that boost content based on the poster's engagement level, automation that wants to spam cannot benefit from this because slowing things down to show as if it was made in real time by a human (with fully non-headless browser etc.) does somewhat defeat t…

I think the key here is that as much as some AI doomsayers don't want it to be the case, and even as much as a lot of people would claim to care, most people won't care enough about whether or not they're consuming AI content or human content to for it to be worthwhile investing all that much effort into stopping automation.

What matters a lot more is to stop abusive behaviour and low quality slop - whether or not that is written by humans or AI.

And working to detect that is a lot more tractable problem than trying to stop automation.

Re: Proving a human wrote something

#84

Detecting Ai use based on "how many keystrokes did you type and when" doesn't solve the problem, because someone can just write a program to mimic human typing in the words. So the issue isn't "did a human write something", it's what the actual content is

Or ask a coding assistant to write one for them...

The bonus, you can use whatever detection method as a judge for a coding assistant loop.

"/goal write a script to pass [some AI detector]"

Re: Proving a human wrote something

#85
post #46

Earlier quoted context omitted.

Here's another similar project: https://writetrack.dev/ There's plenty of microbehavioral analysis we can do that is initially effective but will get bypassed (with GANs being the purest way, or something more domain-specific). You could imagine a livestream that's permanently published somewhere. But the verification of the livestream takes longer than reading the piece itself (and is itself vulnerable to faking). P…

You can read ahead as to how this would go by looking at the CAPTCHA world, which unbeknownst to a lot of people left behind "click this image" as the actual test a long time ago and does a lot of behavioral analysis of mouse motions and stuff. Which is itself a constant arms race. Which I would still characterize as "advantage attacker" with regard to that aspect of it, CAPTCHAs still have at least some utility more…

And yet, for a site I use regularly, I keep getting the CAPTCHA, while Claude Code has automated the hell out of my most important interactions with the site using Playwright, and somehow never triggers the CAPTCHA...

Re: Proving a human wrote something

#86

> “I didn’t think anyone would care” prevented me from writing, though A few years ago I started writing Twitter threads [0]. A few weeks ago I passed 200 total threads. When I started writing them, my thought process was: "Is anyone going to be interested in my stories/ideas??" Dear HN comment reader, I can 100% assure you of two things: 1. If you write things, at least one person will read them. 2. It is VERY hard…

Being able to distill a story down into bite sized chunks sounds like an optimization for the younger generation, with their 20 second attention span. It's not something I necessarily associate with "good writing". Or does the skill transcend the medium and help you write better long form content as well?

Doesn't it boil down to the old "you don't understand it unless you're able to explain it to a 5yo"?

Because if you can do that, you'll also be able to express yourself within character limits

Also - purely because you brought generations up: Twitter is a millennial product. the brainrot Gen is mostly Alpha, and some zoomers.

Re: Proving a human wrote something

#87
This connects with something I've always wanted to have as a "metadata" signal provided with any text. I'm thinking of some low-key information channel like text background color that would communicate how much time it took me to produce this sentence. Darker text used for passages that are "highly polished" (a lot of research and edits where made to fact-check and fine-tune the content and the wording).

Perhaps there might be some "metric" that can be calculated automatically from git commit history, but it would be nicer if the author can manually allocate "weight" to each passage of the text, based on what they perceive is most important (read the dark regions for the the deepest insights).

--

Use Case 1: Emphasis in office communications. When employee A communicates with employee B, they can use the text background to communicate the energy invested by employee A to produce the email/report/memo/doc in question.

Example 1: Developer spent 1 month rewriting the login system to a clean, feature-identical, drop-in replacement Auth API. When he announces to the CTO, the text is in solid grey, communicating the number of hours that went into this project and hence it's importance (better read all the details). dark highlight = I invested a lot of energy in this; I know you're a business person now, but I need you to read this and act on it!

(current alternative way to "highlight" actions is to link them to financial benefits for the company, e.g. the new Auth API will save us hours of dev time and allow us to integrate with platforms X, Y, Z, but I think the "number of hours I put into this" would be a good metric to show as well. It's proximal to the developer's emotional investment in the thing)

--

Use Case 2: Emphasize key ideas in educational text. Educators can communicate the relative importance of different parts of the text. Instead of "energy that went into creating this text," the more useful signal would be to tell readers how much mental energy it will take them to understand a given concept. A sentence that explains some key concept can be shown in dark background to say "this is deep" and also "this is worth learning about specifically within this section." Example 2: An author explains a sequence of 10 complicated steps that are part of some process, and indicates by text background color which steps are important and which are just technical minutiae.

* I write math and science textbooks, and I'd love to have such a sidechannel to the reader to communicate "importance" ... I hack by marking some key stences as bold, but it could be better.

--

Use Case 3: Ethical labelling of genAI output. It is now commonplace within the academic world to require disclosure of genAI use when creating scientific publications and other docs. This is a good practice, but it leaves too much degrees of freedom to the author about the level of disclosure they make. A word-for-word provenance metadata "channel" in parallel with the final text of the document could be a very useful thing to have (in an ideal world). In the real world, few academic would admit to heavily leaning on genAI, but at least we can shame them for not providing the "detailed provenance" track along with their text.

In contrast, people who are using genAI unashamedly (half the population) could prove how un-ashamed of their genAI usage they are by specifying "All genAI" in the "provenance channel" for all the posts they publish. If you like the SLOP or you think you can control the SLOP, I won't judge you, but please let me know so I read the text differently...

Example 3: Author A asks editor E to review a book draft. The manuscript clearly shows which parts of the book were written by the author and which parts are genAI. The editor knows which parts to focus their attention on (what the author is saying), and which parts are just filler. For bonus points, the author could also disclose the harness+context+prompt they used to produce the text (like a view source affordance that comes with any genAI text passage).

Re: Proving a human wrote something

#88

> how do you prove a human wrote something? Forget about the why or the value in it, just: how? > Semoi is a plugin (currently only available for Obsidian) which tracks the length of time it took for a document to be written up Trying to mechanistically prove that a human created some content as opposed to ai, in the age of LLMs and style transfer when you can just ask for something to be written in the style of Mark…

> is going to be short lived because someone will train an AI on a corpus of human keystrokes and get a replication You can remotely attest the input devices. You can do it anonymously (long story, but doable) and without requiring some kind of pre-signed image for the while OS. (The OS passes through recent-input attestations.)

In a world where none of the existing USB/bluetooth keyboards work, perhaps.

Count me out.

Post reply on HN