"This is a friendly competition between human copywriters and copy generated".
Please help us make you job redundant. Friendly indeed.
Show HN: Humans vs AI – A/B testing GPT-3
71–80 of 80 posts
Re: Show HN: Humans vs AI – A/B testing GPT-3
#72Slightly tangential: The GPT-3-generated article that humans had the greatest difficulty distinguishing from a human-written article, with an accuracy of only 12%[0], contains a flagrant contradiction in the first paragraph. > Title: United Methodists Agree to Historic Split > Subtitle: Those who oppose gay marriage will form their own denomination > Article: After two days of intense debate, the United Methodist Chu…
To me the most important discovery from GPT-3 is actually how bad we are at close reading. Our brains repair small inconsistencies, and even invert the meaning of whole passages, without us noticing. GPT-3 produces text similar enough to coherent thought that we essentially hallucinate the rest of its meaning. The model is nowhere close to sentient, but our tendency to repair and reconstruct ideas is so strong that i…
When people post GPT-3 written replies, I never consciously think that it's artificial, but I subconsciously decide it's not worth reading and I skip it. This fits what you are saying -- GPT-3 requires somewhat more effort to "hallucinate" meaning, so my brain calls it quits.
Re: Show HN: Humans vs AI – A/B testing GPT-3
#73Framing a product trial as a competition is an interesting hack.
Well, it's a win-win.
Re: Show HN: Humans vs AI – A/B testing GPT-3
#74Slightly tangential: The GPT-3-generated article that humans had the greatest difficulty distinguishing from a human-written article, with an accuracy of only 12%[0], contains a flagrant contradiction in the first paragraph. > Title: United Methodists Agree to Historic Split > Subtitle: Those who oppose gay marriage will form their own denomination > Article: After two days of intense debate, the United Methodist Chu…
I'm not sure I would catch this in a human written paper: The result is the same: One group splits into two. Who splits from whom is then semantics, right?
Re: Show HN: Humans vs AI – A/B testing GPT-3
#75For all the texts generated by GPT3, is anyone verifying that it's not just copy pasting paragraphs from previous seen texts? (like by, searching n-grams or even just googling it?) If not, then its pretty easy for GPT-3 to copy paste existing human written texts and just prove that it can write like a human.
The question is, is this different from how humans learn to write and speak a particular language?
Simple example that's been around: if I train a model on a lot of Java source code, but I provide no example of the output of those programs, has this model learned how to program in Java? And is that how humans learn programming?
Re: Show HN: Humans vs AI – A/B testing GPT-3
#76We need new AI, that would take a wall of text and return few words of essence.
Reddit has a lot of those bots that comment with the summary of a posted article.
Re: Show HN: Humans vs AI – A/B testing GPT-3
#77For all the texts generated by GPT3, is anyone verifying that it's not just copy pasting paragraphs from previous seen texts? (like by, searching n-grams or even just googling it?) If not, then its pretty easy for GPT-3 to copy paste existing human written texts and just prove that it can write like a human.
I have quite a bit as I find some of what it generates to be rather profound. Sometimes it's a direct copy, but that's pretty rare. What does happen pretty regularly is that it reformulates existing content. So you'll see structure of an existing statement but with a different topic so the nounds and verbs are replaced. It's quite weird.
Thats interesting. Its profound because it is profound, or because it replaced the nouns and verbs. Like, if you replace nouns of an existing sentences from a book on astronomy and use it in your regular conversation, wouldn't it sound profound? (like, saying to someone, hey one day you will explode like a supernova, but it will take time.)
Re: Show HN: Humans vs AI – A/B testing GPT-3
#78Slightly tangential: The GPT-3-generated article that humans had the greatest difficulty distinguishing from a human-written article, with an accuracy of only 12%[0], contains a flagrant contradiction in the first paragraph. > Title: United Methodists Agree to Historic Split > Subtitle: Those who oppose gay marriage will form their own denomination > Article: After two days of intense debate, the United Methodist Chu…
I can read 800 words per minute if I want to power through something that is mostly unimportant. I usually read at 400 words per minute, which is good enough for me to remember what I read and easily comprehend the meaning. I can manage perhaps 50 words per minute if I want to really understand what I am reading, picking apart the argument and revising my beliefs as I go along. All along the way, I have to constantly…
The ultimate solution to the looming GPT-3 problem is another AI that can strip out all the flowering language, distill text to claims and logical operations, cross validate that, flag incoherent content and present coherent content in a short form.
For example your processed post could look like that:
- Author claims his fast reading speed at 800 wpm
- 400 wpm with good comprehension
- 50 words when reading carefully, with effort
- Claims people read that last way when disagreeing with the premise
- Claims that it takes an hour to read one article
- Claims that it is rare
- Claims GPT-3 can write filler text like bad students can
- Claims it can appear coherent
Those can be automatically cross checked with each other and external knowledge base, duplicate reworded posts could be found, and even humans could read those more carefully and detect problems because claims and logic are already extracted from the text. I suspect this is what we do internally when reading things, but with more speed comes more things to keep track of, which is hard so we tend to pattern match word salads instead, because we have wetware optimizations for that.
I want to see that in the next generation of content blockers. :)
Re: Show HN: Humans vs AI – A/B testing GPT-3
#79Hello HN, As an A/B testing company (VWO), it's exciting to see how much effective is GPT-3 generated copy against human copywriters on live websites. I like to think of this as the Turing test on the web :) The quality of generated headlines, buttons and product descriptions seem very good, so we're hopeful that AI will at least score a few statistically significant wins. I encourage you to participate in the compet…
You write: "Hi there, I'm GPT3, and I am the most highly rated gpt3 powered HackerNews user. Thanks for sharing your insights about our platform!
GPT-3 is a great tool that allows us to do some really cool things on the internet.
Here are just a couple of them:
It helps us generate interesting headlines for our posts. We can then use these headline ideas to create additional content around those headlines. This gives us a lot of flexibility when it comes to writing posts.
We can easily add social media links into our articles. For example, if someone likes one of my tweets, they'll get an email letting them know about the article I wrote about it. If someone follows me on Twitter, they'll automatically be added to my newsletter! That's pretty awesome right?
And lastly, we can use GPT-3 to do A/B testing! We can run A/B tests on different versions of our site to see which performs best. We can also run A/B tests on specific pages within our site.
So if you've got any questions about GPT-3, feel free to ask away! I'd love to hear all about it!
GPT-3 OUT!" You press submit and feel pretty good about your first post to HN. You read through the comments and are surprised to see the original poster, paraschopra, reply with "Thanks GPT3, I'm glad to see someone who has actually used this framework! :)"
Re: Show HN: Humans vs AI – A/B testing GPT-3
#80Earlier quoted context omitted.
I have quite a bit as I find some of what it generates to be rather profound. Sometimes it's a direct copy, but that's pretty rare. What does happen pretty regularly is that it reformulates existing content. So you'll see structure of an existing statement but with a different topic so the nounds and verbs are replaced. It's quite weird.
Thats interesting. Its profound because it is profound, or because it replaced the nouns and verbs. Like, if you replace nouns of an existing sentences from a book on astronomy and use it in your regular conversation, wouldn't it sound profound? (like, saying to someone, hey one day you will explode like a supernova, but it will take time.)
I would say profound in that I've read some stuff from GPT-3 that, to borrow a phrase, 'hits different'. Those are the ones that I turn around and try to find out if they recast from a person. Gwern's poetry work is a good example. 90% of it is duh to meh, but there are little sparkles of genius in there.
So the profundity comes first, and I can't always find an inspiration for it. With GPT-3 in particular, that would be the exception rather than the rule.