Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI poetry is indistinguishable from human poetry and is rated more favorably

nature.com

Post reply on HN
Print view View original

141–150 of 217 posts

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#141

Post by 8note »

ilaksh wrote: post #97
ubertaco wrote: post #18

Poetry, like humor, involves the use of the reader's expectations, but is typically most effective when subverting those expectations. There's a lot of bad poetry in the world that just follows readers' expectations. I should know, I've written some of it. Unfortunately, I'd suspect that most readers' understanding of poetry lacks that crucial element of subversion, and so an LLM – which mostly just spits out the mos…

One could literally take Claude 3.5 Sonnet New or o1-preview and disprove this in an hour or two just by prompting the AI to try to exhibit the type of poetry you want and then maybe asking it to do a little bit of automated critique and refinement. You can also experiment with having a higher temperature, (maybe just for the first draft). You claim that LLMs can't make poetry like that. I bet they can if you just as…

if im reading the gp properly, theyre saying that an llm isnt capable od inventing new poetry forms.

of its easy, can you provide some poetry forms youve coaxed sonnet to create, with some exemplary poems in the form?

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#142

Post by archagon »

glitchc wrote: post #69
pclmulqdq wrote: post #29

Earlier quoted context omitted.

This is a great example of a clearly very bad poem that would sound "good enough" to people. Some things rhyme, but there's very clearly no plan.

That definition covers 90% of popular hip-hop, pop and r&b.

I would argue that 99% of popular songs have fairly poor quality lyrics (though it doesn't really matter with regard to artistic merit of the finished product). Singer-songwriters in folk-adjacent genres such as Joni Mitchell, Sufjan Stevens, and Don McLean come closest to actual poetry IMHO.

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#143

Post by alfalfasprout »

gcau wrote: post #122
lm28469 wrote: post #52

Earlier quoted context omitted.

The vast majority of people seem to prefer Avengers 17 over any cinematic masterpiece, the latest drake song would be better rated than a Tchaikovsky... We should let them play and worship chat gpt if that's what they want to waste their time on

I don't understand the logic of calling superhero movies lesser/unserious like this, it's very snobby. Movies and music are made to be entertaining, the avengers is more entertaining than your "arthouse cinematic masterpiece that nobody likes but it's just because they aren't smart enough to understand it". It's also lazy and ignorant to ignore the sheer manpower that goes into making a movie like that.

"Movies and music are made to be entertaining"

In your opinion, perhaps. Other films are made to be provocative-- to make you think or reflect. Certainly, a lot of the "arthouse cinematic masterpieces" aim for that as a goal rather than purely entertainment.

You're arguing against a strawman here... nobody is saying making an avengers movie is low effort. Certain aspects of an avengers movie though require less effort.

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#144

Post by I-M-S »

sirspacey wrote: post #139
lm28469 wrote: post #126

Earlier quoted context omitted.

> your "arthouse cinematic masterpiece that nobody likes You're reading way too much into my comment. Any block buster from the 80/90s absolutely shits on 90% of block busters released today. I'm not talking about obscure 1950s czechoslovak cinema here... > ignore the sheet manpower that goes into making a movie like that. A lot of work doesn't make something good, especially when cgi quality actually gets worse year…

Film is such a nascent art form. The 90s as “peak blockbuster action” is a valid stance on taste but hard to defend as superior to all that came after. Christopher Nolan’s Dark Knight is leagues aways from the 90s Batman, as an auteur friendly and obvious comparison. Pixar another on the animation front. There have been great films made in every era, but the trend towards tighter writing, more legible and compelling…

I hope you're referring to Joel Schumacher's kitschy drivel, and not to Tim Burton's masterpieces (both of which are IMO vastly superior to Nolan's take on the subject).

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#145

Post by ilaksh »

8note wrote: post #141
ilaksh wrote: post #97

Earlier quoted context omitted.

One could literally take Claude 3.5 Sonnet New or o1-preview and disprove this in an hour or two just by prompting the AI to try to exhibit the type of poetry you want and then maybe asking it to do a little bit of automated critique and refinement. You can also experiment with having a higher temperature, (maybe just for the first draft). You claim that LLMs can't make poetry like that. I bet they can if you just as…

if im reading the gp properly, theyre saying that an llm isnt capable od inventing new poetry forms. of its easy, can you provide some poetry forms youve coaxed sonnet to create, with some exemplary poems in the form?

What he said was basically that it just couldn't create unexpected verses or break form. Since supposedly it can only do the most probable token -- but that's not how sampling works unless you use temperature 0. And it can easily be instructed to break from a strict form (which would create a new variation of the form) for effect if it made sense.

You could also ask it to create a new form and it could. I don't work for you so I don't have to create examples, but anyone who has used the latest SOTA models for any amount of time knows this capability is expected, and if you were really interested then you would try it. If you feel the result isn't very good, ask it to improve it.

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#146

Post by lukev »

EMIRELADERO wrote: post #130
lukev wrote: post #106

Earlier quoted context omitted.

That may be how most people see art, but it is not how artists see art. Neither side is "right" or has a monopoly on the definition of art. You can't tell someone else how to think or feel about something.

> That may be how most people see art ...which is all that matters in this specific context as we're discussing generative AI's impact on society and what could happen in the future.

Except that the direction of art is usually influenced by the artists, without too much calibration to what the uninformed public wants.

It’s actually a really interesting question if this affects the market, at all. Because if AI can generate “art” which satisfies the public, but not artists, will the public just go for that, instead?

Or is it the case that the public never really cared for or consumes that art form, and that the entire market for art is to people who (to at least some degree) specialize in that form?

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#147

Post by jart »

HPsquared wrote: post #50
aithrowawaycomm wrote: post #34

Ernest Davis has an outstanding and definitive rebuttal of these claims: https://cs.nyu.edu/~davise/papers/GPT-Poetry.pdf The basic problem is that GPT generates easy poetry, and the authors were comparing to difficult human poets like Walt Whitman and Emily Dickinson, and rating using a bunch of people who don't particuarly like poetry. If you actually do like poetry, comparing PlathGPT to Sylvia Plath is so offensi…

I don't agree that it's possible to describe a piece of poetry as "objectively" better or worse.

You can if the assignment is to write something in iambic pentameter.

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#148

Post by bccdee »

pessimizer wrote: post #101
bccdee wrote: post #80

Earlier quoted context omitted.

> I don't see how the authors are dishonest about it, or that the rebuttal refutes any of the actual claims made. Suppose I prepared a glass of lemonade and purchased a bottle of vintage wine—one held in high regard by connoisseurs. Then I found a set of random bystanders and surveyed them on which drink they preferred, and they generally preferred the lemonade. Do you see how it would be dishonest of me to treat thi…

> Suppose I prepared a glass of lemonade and purchased a bottle of vintage wine—one held in high regard by connoisseurs. I would tell you that there have been results about the blind testing of wines held in high regard by connoisseurs that might make you not want to choose that for a comparison.

The blind tasting studies prove that connoisseurs can't discern the price of wine by taste. They can tell whether or not they like it perfectly well. A good bottle, not an expensive one.

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#149

Post by Spivak »

gcau wrote: post #120
aithrowawaycomm wrote: post #57

Earlier quoted context omitted.

I would say Davis's definition of "objectively better" here is "nobody who reads these poems carefully could possibly conclude that this AI crap is better than Walt Whitman, the only explanation is Walt Whitman is so difficult that the raters didn't read it carefully." The Nature paper is making a bold and anti-humanist claim right in the headline, laundering bullshit with bad data, without considering how poorly-def…

>This data really is awful because the subjects aren't interested in reading difficult poetry. If the results were the opposite, would the data still be awful?

That's my litmus test for whether I'm confirmation biasing myself.

Obviously this study is flawed and the results are garbage! But if the study had concluded the opposite then I knew it!

Re: AI poetry is indistinguishable from human poetry and is rated more favorably

#150

Post by jart »

ilaksh wrote: post #97
ubertaco wrote: post #18

Poetry, like humor, involves the use of the reader's expectations, but is typically most effective when subverting those expectations. There's a lot of bad poetry in the world that just follows readers' expectations. I should know, I've written some of it. Unfortunately, I'd suspect that most readers' understanding of poetry lacks that crucial element of subversion, and so an LLM – which mostly just spits out the mos…

One could literally take Claude 3.5 Sonnet New or o1-preview and disprove this in an hour or two just by prompting the AI to try to exhibit the type of poetry you want and then maybe asking it to do a little bit of automated critique and refinement. You can also experiment with having a higher temperature, (maybe just for the first draft). You claim that LLMs can't make poetry like that. I bet they can if you just as…

They could, but they probably won't. Poems like GP are basically using the power of emotional manipulation for good, and companies like Anthropic try very hard to prevent Claude from having that capability.
Post reply on HN