Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI vs. Professional Authors Results

mark---lawrence.blogspot.com

Post reply on HN
Print view View original

61–70 of 103 posts

Re: AI vs. Professional Authors Results

#61

Post by biffles »

unignorant wrote: post #53

Here are my notes and guesses on the stories in case people here find it interesting. Like some others in the blog post comments I got 6/8 right: 1.) probably human, low on style but a solid twist (CORRECT) 2.) interesting imagery but some continuity issues, maybe AI (INCORRECT) 3.) more a scene than a story, highly confident is AI given style (CORRECT) 4.) style could go either way, maybe human given some successful…

Could you expand on your point re more sophisticated prompting?

I have found it hard to replicate high quality human-written prose and was a bit surprised by the results of this test. To me, AI fiction (and most AI writing in general) has a certain “smell” that becomes obvious after enough exposure to it. And yet I scored worse than you did on the test, so what do I know…

Re: AI vs. Professional Authors Results

#62

Post by thenoblesunfish »

waltbosz wrote: post #48

There is an Asimov story called "Someday" in which a toy computer called a Bard generates random fairy tales and reads them to children. In the story two children try to hack their Bard, to make it tell more interesting modern stories, by feeding it a new vocabulary of modern words. In the end, it just generates the same old fairy tale plots using the new words it has learned. I really feel like that story embodies t…

Is this why the Google product (now just called Gemini) was called that?

Re: AI vs. Professional Authors Results

#63

Post by unignorant »

codechicago277 wrote: post #55
unignorant wrote: post #53

Here are my notes and guesses on the stories in case people here find it interesting. Like some others in the blog post comments I got 6/8 right: 1.) probably human, low on style but a solid twist (CORRECT) 2.) interesting imagery but some continuity issues, maybe AI (INCORRECT) 3.) more a scene than a story, highly confident is AI given style (CORRECT) 4.) style could go either way, maybe human given some successful…

I had similar results, and story 4 is so trope heavy I wonder if it’s just an amalgamation of similar stories. The human stories all felt original, where none of the AI ones did.

I'm not sure I agree that the human stories felt original. I was pretty unimpressed with all of the stories except maybe 6, and even that one dealt in some common tropes. 5 had fewer tropes than 6 (and maybe as a result of that received the highest average scores from his readers) but I could tell from the style it was AI

Re: AI vs. Professional Authors Results

#64

Post by dwd »

npinsker wrote: post #54
dwd wrote: post #8

Earlier quoted context omitted.

Big fan of Janny Wurts, particularly the Empire series she co-wrote with Raymond E Feist. Very surprised she was outed as an AI, though she is the only author I've ever had to pull out the dictionary as she used a word I had never seen and wanted to be sure of the meaning in the context.

I was very (wrongly) confident she was AI. I thought her prose was bland and repetitive and lacking melody, and she was the only human author to explicitly mention demons in the first paragraph, which is something that happens when you prompt an AI for a demon story. The twist had potential, but to me wasn’t executed as well as it could have been. I was hoping for some stronger irony — e.g. if the mainlanders had pus…

The twist was the one thing that made me think it was human written. I quite liked her story after the second read. But sad to say, I actually preferred most of the AI ones over the rest.

Re: AI vs. Professional Authors Results

#65

Post by breuleux »

unignorant wrote: post #53

Here are my notes and guesses on the stories in case people here find it interesting. Like some others in the blog post comments I got 6/8 right: 1.) probably human, low on style but a solid twist (CORRECT) 2.) interesting imagery but some continuity issues, maybe AI (INCORRECT) 3.) more a scene than a story, highly confident is AI given style (CORRECT) 4.) style could go either way, maybe human given some successful…

The only one I was fairly sure was human was #6, and that was the only one I kinda enjoyed. In any case, as someone who reads a good deal, I agree. I didn't think any of the stories was particularly great (not enough to bother ranking them, beyond favourite) so I don't care all that much about the result.

> AI can do much better than what is posted above for flash if given more sophisticated prompting.

How sophisticated, compared to just writing the thing yourself?

Re: AI vs. Professional Authors Results

#66

Post by exmadscientist »

Aeolun wrote: post #37
exmadscientist wrote: post #31

Earlier quoted context omitted.

Perhaps on a relative scale "most" fiction doesn't carry any sort of deeper meaning, but if you look at things like "Hugo or Nebula Award nominees" (to pluck out the SF/F genre as a category), I'd say that almost every single one of them, going back all those decades, has something more to say than just their straightforward text. And unless reading is your day job or only hobby, that's a massive, massive corpus of i…

Sure, but that more than surface level meaning comes out in the story, not often in the specific way the sentences are written (I acknowledge those exist, I just don’t consider them the majority). Also, you say you don’t understand why anyone would be interested in the AI slop. But from the article we learn that one is indistinguishable from the other (apparently even to the one professional author that tried)

I was disappointed that the results shown didn't break things down a bit more between star ratings given and authorship guesses. I didn't think any of these stories were amazing flash fiction, and I think that's relevant here. I'm curious to know what people who liked them all, or at least really liked one of them, had to say on judging AI-vs-human.

Re: AI vs. Professional Authors Results

#67

Post by taneq »

vunderba wrote: post #21

The story plot was "Meeting a dragon". As both a human and a writer, challenge accepted: Long ago, there lived a golden dragon whose fractal-like scales gleamed in the glow of the morning in her cave. She was known for her kindness, and many came not with sword or spear, but with humble requests - for you see, it was widely believed that the mystical scales of a dragon would heal illness, cure ailments, and provide f…

No good deed goes unpunished!

Re: AI vs. Professional Authors Results

#68

Post by unignorant »

biffles wrote: post #61
unignorant wrote: post #53

Here are my notes and guesses on the stories in case people here find it interesting. Like some others in the blog post comments I got 6/8 right: 1.) probably human, low on style but a solid twist (CORRECT) 2.) interesting imagery but some continuity issues, maybe AI (INCORRECT) 3.) more a scene than a story, highly confident is AI given style (CORRECT) 4.) style could go either way, maybe human given some successful…

Could you expand on your point re more sophisticated prompting? I have found it hard to replicate high quality human-written prose and was a bit surprised by the results of this test. To me, AI fiction (and most AI writing in general) has a certain “smell” that becomes obvious after enough exposure to it. And yet I scored worse than you did on the test, so what do I know…

For flash you can get much better results by asking the system to first generate a detailed scaffold. Here's an example of some metadata you might try to generate before actually writing the story: genres the story should fit into; pov of the story; high level structure of the story; list of characters in the story along with significant details; themes and topics present in the story; detailed style notes

From there you have a second prompt to generate a story that follows those details. You can also generate many candidates and have another model instance rate the stories based on both general literary criteria and how well the fit the prompt, then you only read the best.

This has produced some work I've been reasonably impressed by, though it's not at the level of the best human flash writers.

Also, one easy way to get stuff that completely avoids the "smell" you're talking about by giving specific guidance on style and perspective (e.g., GPT-5 Thinking can do "literary stream-of-consciousness 1st person teenage perspective" reasonably well and will not sound at all like typical model writing).

Re: AI vs. Professional Authors Results

#69

Post by ethan_smith »

dwd wrote: post #50

The best fantasy/sci-fi literature involves a lot of world building. For some, the world building came first and the stories were an offshoot of that. Tolkien needed a world and stories to bring life to the languages he was inventing. Raymond E Feist's Midkemia was a massive collaborative effort for a RPG world. He has stated: "I don't write fantasy; I write historical novels about an imaginary place. At least that's…

Multi-agent LLM systems with persistent memory are actually making significant progress on world-building coherence, maintaining consistency across thousands of interactions while incrementally developing complex fictional universes.

Re: AI vs. Professional Authors Results

#70

Post by famouswaffles »

I put the basic prompt to 5 medium thinking in the API (because most of the writing gains seems to be tucked away in the reasoning mode and because you can't trust the router for this stuff) and this is what i got. It's not flash fiction, more a short story, but i'm impressed.

https://pastebin.com/huGhbX7u

Post reply on HN