Live data from Hacker News

Show HN: Humans vs AI – A/B testing GPT-3

vwo.com

21–30 of 80 posts

Re: Show HN: Humans vs AI – A/B testing GPT-3

#21
I'm a little disappointed that the landing page for this challenge itself doesn't seem to be a live a/b test with live results for how much submissions are garnered through the human version of the page versus the gpt-3 one.

Fun challenge though !

Re: Show HN: Humans vs AI – A/B testing GPT-3

#22

Interesting... honestly having seen some of GPT-3's output I'd be curious how well it performs here. One of the things that I think can still give GPT-3 away (GPT-2 as well) ... is that even if the text feels real, it lacks a deep emotional cohesion. Sometimes this can feel like an advanced word salad generator, some poetry can be recognizable this way, because some GPT poems can seem 90% real, but when compared to a…

It will work for this use case. In any case, most probably a human will select from a bunch of generated texts. Whether the texts are generated by a copywriter or GPT-3 will be the difference. So for small texts like heading and CTA buttons this should work. Longer texts are a different story though.

More than A/B testing this might be a better fit for web site building tools like wix.com and and webflow.com

Re: Show HN: Humans vs AI – A/B testing GPT-3

#23

Slightly tangential: The GPT-3-generated article that humans had the greatest difficulty distinguishing from a human-written article, with an accuracy of only 12%[0], contains a flagrant contradiction in the first paragraph. > Title: United Methodists Agree to Historic Split > Subtitle: Those who oppose gay marriage will form their own denomination > Article: After two days of intense debate, the United Methodist Chu…

Maybe we're so used to "denominations" meaning the exact opposite or at least something totally unrelated to what the party/group/association/lobby... really is that it's not really shocking anymore.

Re: Show HN: Humans vs AI – A/B testing GPT-3

#24

Slightly tangential: The GPT-3-generated article that humans had the greatest difficulty distinguishing from a human-written article, with an accuracy of only 12%[0], contains a flagrant contradiction in the first paragraph. > Title: United Methodists Agree to Historic Split > Subtitle: Those who oppose gay marriage will form their own denomination > Article: After two days of intense debate, the United Methodist Chu…

I had to read it twice to notice the contradiction.

I think the cause is more that some religious group i dont care about having a schism isn't very interesting. Maybe i care that they're being homophobic, but i definitely don't actually care which side is forking and which side is staying as the original group. So I don't fully pay attention to the details unless i really force myself.

Re: Show HN: Humans vs AI – A/B testing GPT-3

#25

Slightly tangential: The GPT-3-generated article that humans had the greatest difficulty distinguishing from a human-written article, with an accuracy of only 12%[0], contains a flagrant contradiction in the first paragraph. > Title: United Methodists Agree to Historic Split > Subtitle: Those who oppose gay marriage will form their own denomination > Article: After two days of intense debate, the United Methodist Chu…

I can't believe I'm defending GPT-3, but...

1) Humans often make logical errors

2) In the religious-split context, who is the original denomination and who is the splitting denomination is inherently fraught/subjective/liable-to-contradiction/open-to-changing-contextualisation, so its not a surprising mistake. The readers may have just assumed it was the author/speaker/washington post getting mixed up or projecting their own opinion, which may have contrasted with that expressed from the reports/attendees of the 'actual conference'.

(plus there's a good chance none of them could give two hoots about, or have any specific knowledge or interest in methodists...i have a religious studies degree and my eyes are already almost glazing over just at the mention of them to be honest :P)

Re: Show HN: Humans vs AI – A/B testing GPT-3

#26

Slightly tangential: The GPT-3-generated article that humans had the greatest difficulty distinguishing from a human-written article, with an accuracy of only 12%[0], contains a flagrant contradiction in the first paragraph. > Title: United Methodists Agree to Historic Split > Subtitle: Those who oppose gay marriage will form their own denomination > Article: After two days of intense debate, the United Methodist Chu…

To me the most important discovery from GPT-3 is actually how bad we are at close reading. Our brains repair small inconsistencies, and even invert the meaning of whole passages, without us noticing. GPT-3 produces text similar enough to coherent thought that we essentially hallucinate the rest of its meaning. The model is nowhere close to sentient, but our tendency to repair and reconstruct ideas is so strong that it doesn’t matter.

Makes you think, how often does this happen with other writing?

Re: Show HN: Humans vs AI – A/B testing GPT-3

#27
I've posted this before, but here's one service I can absolutely foresee:

Automated job applications

and

Automated job listings

Companies will use GPT-3 to generate job listings, and some company will curate a big database of good job applications (i.e those that have landed someone a job), and make a service where you feed in the listing, and out comes job application / letter.

Re: Show HN: Humans vs AI – A/B testing GPT-3

#28
post #6

I wouldn't be surprised if the result of the next election in a country would be decided by a Transformer based deep neural network. The dictator in my country is already paying stupid people to write stupid comments, GPT-3 is already over their level.

cf "Franchise", Isaac Asimov, 1955: https://en.wikipedia.org/wiki/Franchise_(short_story)

Re: Show HN: Humans vs AI – A/B testing GPT-3

#29

I've posted this before, but here's one service I can absolutely foresee: Automated job applications and Automated job listings Companies will use GPT-3 to generate job listings, and some company will curate a big database of good job applications (i.e those that have landed someone a job), and make a service where you feed in the listing, and out comes job application / letter.

Interesting thought. I would assume that job listings is easier to generate because it's a minimum threshold task, whereas job applications would be orders of magnitude harder because having a good application and being accepted to a job is weakly correlated at most.

Re: Show HN: Humans vs AI – A/B testing GPT-3

#30
post #5

I feel like GPT-3 is going to unleash the next wave of SEO spam. Content spinning will be taken to the next level, will Google be able to detect GPT-3 content?

It’s already happening for years. Steps: 1) identify competitor blogs 2) scrape all their posts 3) Run it through nlp to rewrite words, phrasing and sentence structures while keeping content. 4) Tidy up using SEO guidelines, linking and keyword research. 5) publish 100s of articles a month as one single person
Post reply on HN