I played with this and it is super interesting (almost made me register a couple domain names!) That said, to me, it once again reinforces the belief that a large factor in GPT-3 amazingness is the taste in prompting/filtering that humans apply to it, that is, it produces a ton of crap that does not catch our eye that we just silently ignore and discard, but we will amplify and share amongst ourselves the output that…
Computers aren't creative; the excitement about GPT-3 is humans projecting something into its output or filtering the small number of bits that appear to make sense, as you say. Neural language models are just recycling bits that humans have said before, they address well the "how to say" part of NLG (Natural Language Generation) but fail with respect to the "what to say" part.
We've been discussing the topic of machine intelligence since the start of CS and points of contention tend to show up in the same places.
Anyway, it occured to me that if given 10 tries, GPT-3 probably produces my comment adequately. I have an uneasy suspicion that it might take fewer tries than that... especially if prompted with "netcan, please respond."
Anyway, the upshot is that for someone like me, on the "if you can't tell the difference" side of the cliche... your side's insistence that machine's can't be creative makes me doubt my own creativity, to the extent that the argument is convincing.