Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaWrite: AI that improves at writing by evolving its own stories

tobysimonds.com

Post reply on HN
Print view View original

61–70 of 162 posts

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#61

Post by vidarh »

saberience wrote: post #58
soulofmischief wrote: post #56

Earlier quoted context omitted.

Not everyone shares your same world view, and some people do want to apply machine intelligence to their writing process. You don't have to participate; ignore AI-generated or AI-assisted content just like you ignore some other thing you don't enjoy that already exists today. But you also don't have to devalue and dismiss the interests of others.

All the people generating AI-assisted writing are all the people that never had enough passion or talent to do it before. If you weren't inclined to write fiction or poetry etc before AI was here to do it for you, you probably shouldn't be doing it now.

That's extremely presumptuous. I've published two novels, and written hundreds of poems over the years (the latter I'm not sure I'll ever publish), and while I will keep writing manually I'd love to have AI tools that'd write all of the things I want to read that doesn't exist, that I don't want to write myself.

I don't get remotely the same things out of reading and writing, so writing those stories myself does not give me the enjoyment I'd want out of reading them.

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#62

Post by vidarh »

riskable wrote: post #54
suddenlybananas wrote: post #6

I can't imagine LLMs are good judges of good writing.

I've pasted whole chapters (of my own writing) into ChatGPT and Claude that I know need drastic improvements. Basically, they were first draft, "get the concept down; don't think too hard" paragraphs with occasional run-on sentences and whatnot. This is my very first novel (ever) so of course the initial draft is going to be bad. Both ChatGPT and Claud always say something like, "a few grammar corrections are needed…

That sounds at least partly like a prompting issue to me. I have no problem getting scathing critiques out of either, by defining the role I want them to take clearly.

Here's part of an initial criticism Claude made of your comment (it also said nice things):

"However, the prose suffers from structural inconsistencies. The opening sentence contains an awkward parenthetical insertion that disrupts flow, and the second sentence uses unclear pronoun reference with "This" and "they." The rhythm varies unpredictably between crisp, direct statements and meandering explanations.

"The vocabulary choices are generally precise—"sycophants" is particularly apt and memorable—though some phrases like "get the concept down; don't think too hard" feel slightly clunky in their construction."

This was the prompt I used:

"Imagine you're a literary critic. Critique the following comment based on use of language and effectiveness of communication only. Don't critique the argument itself:" followed by your comment.

"Image you're a ..." or "Act as a ..." tends to make a huge difference in the kind of output you get. If you put it in the role of a critic that people expect to be tough, you're less likely to get sycophantic responses, at least in my experience.

(If you want to see it get brutal, follow up the first response with a "be harsher" - it got unpleasantly savage)

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#63

Post by suddenlybananas »

vidarh wrote: post #62
riskable wrote: post #54

Earlier quoted context omitted.

I've pasted whole chapters (of my own writing) into ChatGPT and Claude that I know need drastic improvements. Basically, they were first draft, "get the concept down; don't think too hard" paragraphs with occasional run-on sentences and whatnot. This is my very first novel (ever) so of course the initial draft is going to be bad. Both ChatGPT and Claud always say something like, "a few grammar corrections are needed…

That sounds at least partly like a prompting issue to me. I have no problem getting scathing critiques out of either, by defining the role I want them to take clearly. Here's part of an initial criticism Claude made of your comment (it also said nice things): "However, the prose suffers from structural inconsistencies. The opening sentence contains an awkward parenthetical insertion that disrupts flow, and the second…

Those critiques are nonsensical; the referents of "this" and "they" are completely obvious in the comment and the "clunky" construction is completely fine. There's also no "this" in the second sentence?

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#64

Post by deadbabe »

sorcerer-mar wrote: post #41
deadbabe wrote: post #15

Earlier quoted context omitted.

You cannot stop people from making the world worse or better. The best you can do is focus on your own life. In time many will say we are lucky to live in a world with so much content, where anything you want to see or read can be spun up in an instant, without labor. And though most will no longer make a living doing some of these content creation activities by hand and brain, you can still rejoice knowing that thos…

> A human who writes or produces art for monetary reasons is only just as bad as AI. Or they're what you call "a professional artist," aka "people who produce art so good that other people are willing to pay for it." Another HN commenter who thinks artfulness is developed over decades and that individual art pieces are made over hundreds of hours out of some charity... Ridiculously ignorant worldview.

> Or they're what you call "a professional artist," aka "people who produce art so good that other people are willing to pay for it."

If this is okay, then why isn’t an AI that produces art so good that other people are willing to pay for it also not okay? They are equivalent.

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#65

Post by deadbabe »

conartist6 wrote: post #19
deadbabe wrote: post #15

Earlier quoted context omitted.

You cannot stop people from making the world worse or better. The best you can do is focus on your own life. In time many will say we are lucky to live in a world with so much content, where anything you want to see or read can be spun up in an instant, without labor. And though most will no longer make a living doing some of these content creation activities by hand and brain, you can still rejoice knowing that thos…

> In time many will say we are lucky to live in a world with so much content, where anything you want to see or read can be spun up in an instant, without labor. Man, you are talking about a world that's not just much worse but apocalyptically gone. In that world, there is no more art, full stop. The completeness and average-ness of stimulation would be the exact equivalent of sensory deprivation.

It seems paradoxical to say there is no more art, when AI’s ability for art generation is infinite.

AI art can be equally stimulating, especially for people who will eventually be born in a time when AI generated art has always existed for them. It is only resisted by those who have lived their whole lives expecting all art to be human generated.

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#66

Post by sorcerer-mar »

deadbabe wrote: post #64
sorcerer-mar wrote: post #41

Earlier quoted context omitted.

> A human who writes or produces art for monetary reasons is only just as bad as AI. Or they're what you call "a professional artist," aka "people who produce art so good that other people are willing to pay for it." Another HN commenter who thinks artfulness is developed over decades and that individual art pieces are made over hundreds of hours out of some charity... Ridiculously ignorant worldview.

> Or they're what you call "a professional artist," aka "people who produce art so good that other people are willing to pay for it." If this is okay, then why isn’t an AI that produces art so good that other people are willing to pay for it also not okay? They are equivalent.

Who said that's not okay?

The problem with AI-produced art is its potential to supplant human art, i.e. to destroy the incentive for any human to gain artistic mastery.

Here's how they're not equivalent: if you take human inputs out of AI, it disappears. If you take AI inputs out of human art, basically nothing changes.

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#67

Post by gabriel666smith »

tamassimond wrote: post #28
gabriel666smith wrote: post #17

I'm struggling a bit to understand the difference between the reported results in the blog post and the examples in the Github. The blog states: > "Alpha Writing demonstrates substantial improvements in story quality when evaluated through pairwise human preferences. Testing with Llama 3.1 8B revealed: 72% preference rate over initial story generations (95 % CI 63 % – 79 %) 62% preference rate over sequential-prompti…

Just to clarify slight misunderstanding the variants without parent ID aren't from the initial batch it just didn't carry over to the next batch. You can see "897ccd25-4776-4077-a9e6-0da34abb32a4" emerges from batch 5. Apologies probably should make this clearer. Appreciate feedback on blog post!

Ah got you! Makes sense, and makes it so much more clear. Thanks. In that case, I totally retract my crit in my prev comment. Appreciate it.

So - just so I completely understand - the variant we're calling 897ccd25-4776-4077-a9e6-0da34abb32a4 emerged during batch 5, and doesn't have a parent in a prior batch? Very interesting to compare iterations.

I currently run some very similar scripts for one of my own workflows. Though I understand making LLMs do 'good creative writing' wasn't necessarily the point here - perhaps solely to prove that LLMs can improve their own work, according to their own (prompted) metric(s) - the blog post is correct to point out that there's a huge limitation around prompt sensitivity (not to mention subjectivity around quality of art).

As a human using LLMs to create work that suits my own (naturally subjective) tastes and preferences, I currently get around this issue by feeding back on variants manually, then having the LLM update its own prompt (much like a cursorrules file, but for prose) based on my feedback, and only then generating new variants, to be fed back on, etc.

It's extremely hard to one-shot-prompt everything you do or do not like in writing, but you can get a really beefy ruleset - which even tiny LLMs are very good at following - incredibly quickly by asking the LLM to iterate on its own instructions in this manner.

Like I said, not sure if your goal is to 'prove improvement is possible' or 'create a useful creative writing assistant' but, if it's the latter, that's the technique that has created the most value for me personally over the last couple of years. Sharing in case that's useful.

Grats on the cool project!

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#68

Post by riskable »

suddenlybananas wrote: post #63
vidarh wrote: post #62

Earlier quoted context omitted.

That sounds at least partly like a prompting issue to me. I have no problem getting scathing critiques out of either, by defining the role I want them to take clearly. Here's part of an initial criticism Claude made of your comment (it also said nice things): "However, the prose suffers from structural inconsistencies. The opening sentence contains an awkward parenthetical insertion that disrupts flow, and the second…

Those critiques are nonsensical; the referents of "this" and "they" are completely obvious in the comment and the "clunky" construction is completely fine. There's also no "this" in the second sentence?

Exactly... What Claude output is what the prompt asked for: Critique. Even if there was nothing wrong at all Claude would still find something to critique—even if it has to hallucinate something wrong. That's it's job: To generate critique.

It's not really doing any sort of "deep analysis" it's just reading the text and comparing it to what it knows about similar texts from its model. It'll then predict/generate the next word based on prior critiques of similar writings. It doesn't really have an understanding of the text at all.

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#69

Post by gabriel666smith »

zorked wrote: post #5

If there is something that I would like AI to never touch, it's that. Please stop making the world worse.

Wow! Why?

Personally, I'm fascinated by the question of what Joyce would have done with SillyTavern. Or Nabokov. Or Burroughs. Or T S Eliot, who incorporated news clippings into Wasteland - which feels, to me, extremely analogous with the way LLMs refract existing text into new patterns.

Re: AlphaWrite: AI that improves at writing by evolving its own stories

#70

Post by riskable »

stevenAthompson wrote: post #53
riskable wrote: post #49

Earlier quoted context omitted.

> AI-generated slop This phrase is kind of interesting to me because it implies that everything AI-generated is "slop". What happens when the AI is generating decent content ? Like, what if we develop AI to the point where the most insightful, funny, or downright useful content is AI-generated? Will we still be calling it, "AI-generated slop"?

Now that the machines are coming for the white collar, the Luddites those same people once mocked are starting to look more reasonable. In the end the intelligence revolution will be a net benefit to society. In the short term there will be untold suffering.

Let me rephrase that: The machines are coming for bullshit jobs. They're so good at generating bullshit that anyone who generates bullshit for a living needs to be worried.

Only problem is that some huge percentage of white collar work is bullshit. It's no secret. We all know it and accept it.

How many of us have spent weeks or months (or years!) of our lives generating documents that end up going into a black hole (e.g. Sharepoint), never to be read by anyone ever? How many of us have generated presentations that only exist to explain to management what they're supposed to already know? How many of us put together spreadsheets, dashboards, or similar in order to visualize data that doesn't need to be visualized?

We spend our days reading and writing emails that ultimately end up being inconsequential. We waste endless amounts of our time in meetings. Days and weeks and months go by where we "did stuff" that ultimately didn't end up being practical for any purpose.

The people that actually get things done are paid the least and looked down upon. Yet they're the ones that are most likely to survive with their jobs after this "AI revolution."

Post reply on HN