Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Detectors Get It Wrong. Writers Are Being Fired Anyway

gizmodo.com

Post reply on HN
Print view View original

141–150 of 189 posts

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#141

Post by CyberDildonics »

perihelions wrote: post #40
emiliobumachar wrote: post #9

Startup idea: online text editor that logs every keystroke and blockchains a hash of all logs every day. If you're accused of AI use, you can pull up the whole painstaking writing process and prove it's real.

That's a dehumanizing system. Have we lost our way, HN? Are we so immersed in the bleakness of tech, it comes so naturally for us, to propose "hey, let's create surveillance machines to perpetually watch people working, for the rest of their productive lives" and it's something we have to pause and think about? Let's not build Hell on Earth for whatever reason it momentarily seems to make business sense.

They were talking about logging your own encrypted keystrokes and being in control of them.

This would be dehumanizing? This means 'hacker news has lost their way'?

Logging your own keystrokes and encrypting it is 'bleakness of tech'? This is a 'surveillance machine'?

What are you talking about?

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#142

Post by AdhemarVandamme »

viraptor wrote: post #34
emiliobumachar wrote: post #9

Startup idea: online text editor that logs every keystroke and blockchains a hash of all logs every day. If you're accused of AI use, you can pull up the whole painstaking writing process and prove it's real.

The blockchain part is silly, because timestamping services exist without it https://www.sectigo.com/resource-library/time-stamping-serve... The rest is silly, because you can emulate the whole writing process by combining backtracking https://arxiv.org/abs/2306.05426 and a rewriting/rewording loop. With not much effort we can make LLM output look incredibly painstaking.

> The blockchain part is silly, because timestamping services exist without it

Yet the timestamping service which I trust the most, is the Blockchain-based one. https://opentimestamps.org/

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#143

Post by vundercind »

oblio wrote: post #112
vundercind wrote: post #95

Earlier quoted context omitted.

Remind you of some entire genres of book? That’s right, business and self-help books! Any of these with an author who’s got actual accomplishments and money before writing the book was almost certainly already ghostwritten from an outline (and so are lots of other books, you’d be surprised, it’s not just these genres). Successful CEOs or people you’ve heard of generally don’t write their own books. Often, they’re ter…

Well, to err is human, to truly screw up you need a computer. We're going to be blasted to smithereens with LLM-generated "80% should be good enough" garbage.

It’s fortunate we have mountains of human-written books, film, television, radio programs, music, and video games from Before AI. Just the good stuff could occupy several lifetimes.

Pity we killed most of the good used book stores already, though.

Also, shame about journalism and maybe also democracy. That’s too bad.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#144

Post by chipotle_coyote »

NoMoreNicksLeft wrote: post #111
add-sub-mul-div wrote: post #71

Earlier quoted context omitted.

In the general internet the reputation of AI writing is that it's writing that's bad/awkward in a way that is often identifiable (by humans) as not having been written by humans. AI detectors are useless, you're right, but for the same reason AI is unreliable in other contexts, not because AI writing is reliably passable.

> in a way that is often identifiable (by humans) as not having been written by humans. You should check out reddit sometime. It's been nearly twenty years (not hyperbole) of everyone accusing everyone else of being a bot/shill. Humans are utterly incapable of detecting such things. They're not even capable of detecting Nigerian prince emails as scams. > not because AI writing is reliably passable. "Newspaper editor"…

> Also, has it not occurred to anyone that deep down in the brainmeat, humans might actually be employing some sort of organic LLM when they engage in writing?

This is a fairly common take, along with the idea that AI image generators are just doing what humans do when they "learn from examples". But I strongly believe it's a fallacy. What generative AI does is analagous to what humans do, but it's still just an analogy. If you want to see this in action, it's better to look at the way generative AI fails than the way it succeeds: when it makes mistakes in text or images, the mistakes are very much not the kind of mistakes that humans make, because the process behind the scenes is very different.

Yes, obviously when humans write, they take into account context and awareness of what words naturally follow other words, but it seems unlikely we've learned to write by subconsciously arranging all the words we've encountered into multidimensional vector space and performing vector math operations to arrive at the next word based on the context window we're subconsciously constructing. We learn to write in a very different way.

It's truly amazing that generative AI writes as well as it does, but we reason about concepts and generative AI reasons about words. Personally, I'm skeptical that the problems LLMs have with "hallucinations" and with creating definitionally median text* can be solved by making LLMs bigger and faster.

*I did see the comment complaining that it's not mathematically accurate to say that LLMs produce average text, but from my understanding of how generative AI works as well as my recent misadventures testing an AI "novel writer," it's a decent approximation of what's going on. Yes, you can say "write X in the style of Y," but "write X but make it way above average" is not actually going to work.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#145

Post by throwaway48476 »

BugsJustFindMe wrote: post #127
bradley13 wrote: post #5

In some cases an AI will make a weird word choice. So do a lot of humans. Sometimes AIs are needlessly wordy. Um...so are a lot of humans. Rinse and repeat. AI detectors are useless. The AIs are training on human writing, so they write fundamentally like humans. How is this not obvious?

But also maybe firing writers who make weird word choices and are needlessly wordy is fine.

Yes please. The art of writing is conveying the most meaning in the fewest words.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#146

Post by boredpudding »

nerdponx wrote: post #108
jeff_vader wrote: post #53

Reminds this current slightly comedic (IMO) situation in my office: a few months ago developers were given access to GitHub's "Copilot Enterprise". Then, a month or so later, organisation also adopted another "AI" product checking pull requests for risks associated with "use of generative AI". And needless to say it does occasionally fail code written without any "generative AI"..

The flipside is that the AI-written code I've seen at work is usually painfully obvious upon human code review. If you need a tool to detect it, either it's good AI-written code, or you have particularly inept code reviewers.

Be careful here about confirmation bias. If you only spot 10% of the AI-written code, you'll still think you see all of it, because a 100% of the ones you spot are indeed AI-written. And the 10% you see, will indeed be painfully obvious.

The ones you don't notice aren't obvious.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#147

Post by NoMoreNicksLeft »

lottin wrote: post #138
NoMoreNicksLeft wrote: post #111

Earlier quoted context omitted.

> in a way that is often identifiable (by humans) as not having been written by humans. You should check out reddit sometime. It's been nearly twenty years (not hyperbole) of everyone accusing everyone else of being a bot/shill. Humans are utterly incapable of detecting such things. They're not even capable of detecting Nigerian prince emails as scams. > not because AI writing is reliably passable. "Newspaper editor"…

I think we have plenty of evidence that humans have the ability to understand, while chatbots lack such an ability. Therefore, I'm inclined to think that we don't employ some sort of organic LLM but something completely different.

I've occasionally seen evidence that some humans seem to sometimes understand. I've learned not to generalize that though.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#148

Post by dartos »

shmel wrote: post #129
dartos wrote: post #107

Earlier quoted context omitted.

Saying they find some “average” is an easy way to explain to a layman that LLMs are statistically based and are guessing and not actually spitting out correct text as you would expect from most other computer programs. That’s why it’s repeated. It’s kind of correct if you squint and it’s easy to understand

What is the correct text anyway? Everything around you is somewhat wrong. Textbooks (statistically all of them) contain errors, scientific papers sometimes contain handwavy bullshit and in rare cases even outright falsified data, human experts can be guessing as well and they are wrong every now and then, programs (again pretty much all of them) contain bugs. It is just the reality. Even very simple ones may require…

> What is the correct text anyway?

Exactly. The fact that language is fuzzy is why LLMs work so well.

The issue is that most people expect computers to not make mistakes. When you write a formula in an excel sheet, the computer doesn’t mess up the math.

The average non tech person knows that humans make mistakes, but are not used to computers making mistakes.

Many people, maybe most, would see an answer generated by a computer program and assume that it’s the correct answer to their question.

In pointing out that LLMs are guessing at what text to write (by saying “average”) you convey that idea in a simplified way.

Trying to argue that “correct” doesn’t mean anything isn’t really useful. You can replace the word “correct” with “practically correct” and nothing about what I said changes.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#149

Post by NoMoreNicksLeft »

chipotle_coyote wrote: post #144
NoMoreNicksLeft wrote: post #111

Earlier quoted context omitted.

> in a way that is often identifiable (by humans) as not having been written by humans. You should check out reddit sometime. It's been nearly twenty years (not hyperbole) of everyone accusing everyone else of being a bot/shill. Humans are utterly incapable of detecting such things. They're not even capable of detecting Nigerian prince emails as scams. > not because AI writing is reliably passable. "Newspaper editor"…

> Also, has it not occurred to anyone that deep down in the brainmeat, humans might actually be employing some sort of organic LLM when they engage in writing? This is a fairly common take, along with the idea that AI image generators are just doing what humans do when they "learn from examples". But I strongly believe it's a fallacy. What generative AI does is analagous to what humans do, but it's still just an anal…

> But I strongly believe it's a fallacy.

Either the LLM is the most efficient way to generate text, or there's some magic algorithm out there that evolution stumbled upon a million years ago that we haven't even managed to see a hint that it exists. In which case, you'd be right, this is a fallacy.

Or, brainmeat can't do it better or more efficiently, and either uses the same techniques or something even worse. The latter seems unlikely, humans still do pretty well at generating text (gold standard, even).

> it's better to look at the way generative AI fails than the way it succeeds: when it makes mistakes in text or images, the mistakes are very much not the kind of mistakes that humans make, because the process behind the scenes is very different.

But are you looking at "mistakes" that are just little faux pas, or the ones where people with dementia, bizarre brain damage, or blipped out on hallucinogens incorrectly compute the next word? The former offer little insight. Poor taste in word choice, lack of eloquency, vulgar inclinations are what they amount to.

> but it seems unlikely we've learned to write by subconsciously arranging all the words we've encountered into multidimensional vector space and performing vector math operations to arrive at the next word

You think I meant that someone learns to do that at 2 years old, rather than that the brain has already evolved with the ability to do vector math operations or some true equivalent? I'm not talking about some pop psych level "subconscious" thing, but an actual honest to god neurological level faculty.

> but we reason about concepts and Wander into Walmart next time, close your eyes briefly and extend your psychic powers out to the whole building, and tell me if you truly believe, deep down in your heart, that the humans in that store are reasoning about concepts even once a week. That many, if not most, reason about concepts even once a month. I dare you, just go some place like that, soak it all in.

Human reason exists, from time to time, here and there. But most human behavior can be adequately simulated without any reason at all.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#150

Post by perihelions »

CyberDildonics wrote: post #141
perihelions wrote: post #40

Earlier quoted context omitted.

That's a dehumanizing system. Have we lost our way, HN? Are we so immersed in the bleakness of tech, it comes so naturally for us, to propose "hey, let's create surveillance machines to perpetually watch people working, for the rest of their productive lives" and it's something we have to pause and think about? Let's not build Hell on Earth for whatever reason it momentarily seems to make business sense.

They were talking about logging your own encrypted keystrokes and being in control of them. This would be dehumanizing? This means 'hacker news has lost their way'? Logging your own keystrokes and encrypting it is 'bleakness of tech'? This is a 'surveillance machine'? What are you talking about?

If you feel compelled to surveil yourself so as not to be arbitrarily fired by an algorithm, I do consider that dystopian; yes. You're not "in control" of data you're expected to turn over to your employer to keep your job. Worse still if these keyloggers become normalized, and they'll shift from being "optional" to "professionally expected" to "mandated".

This (IMHO) is an example of an attempt at a technical solution for a purely social problem—the problem that employers are permitted to make arbitrary firing decisions on the basis of an opaque algorithm that makes untraceable errors. Technical solutions are not the answer to this. There should be legally-mandated presumptions in favor of the worker—presumptions in the direction of innocence, privacy, and dignity.

This stuff's already illegal on several levels, in some of the more pro-worker countries. It's illegal to make hiring/firing decisions solely on the basis of an algorithm output (EU-wide, IIRC?). And in several EU countries it's illegal to have surveillance cameras pointed at workers without an exceptional reason—and it's not something a worker can consent/opt-in to, it's an unwaivable right. I believe—well, I hope—the same laws extend to software surveillance like keyloggers.

Post reply on HN