Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The computers are getting better at writing

newyorker.com

Post reply on HN
Print view View original

151–160 of 165 posts

Re: The computers are getting better at writing

#151

Post by YeGoblynQueenne »

minimaxir wrote: post #149
YeGoblynQueenne wrote: post #148

Earlier quoted context omitted.

Sorry, you said on pageload, I thought you meant when the page is loaded :) On the cards you posted to reddit: a) "Keening Vythat", "Tombsis Satyr", "Quarpling Barrier". These are typical nonsense-nonsense names (as opposed to the fantasy-nonsense names found on M:tG cards) that are typical, in my experience, of card names generated by language models trained at the character level. M:tG fantasy-nonsense names, like…

Again, this (and my other text generation projects) are optimizing more for humor/chaos than accuracy, and there is a bolded important notice before the text emphasizing that the results may not be legal. Per the other comments on that Reddit thread, that appears to be the correct approach as the crazier-but-barely-legal cards are what resonate the best (the "bad" RoboRosewater cards were the most popular, not the go…

I'm not arguing a "No True Scotsman". I offered criticism to help you improve your work, as criticism should do. I'm sorry that you took it the wrong way.

Re: The computers are getting better at writing

#152

Post by nickpinkston »

dang wrote: post #145
nickpinkston wrote: post #129

Earlier quoted context omitted.

I wouldn't say it's shallow. I think the New Yorker's editorial board / style guide does a disservice in explaining science / engineering concepts. I just delivered my critique via sarcasm.

Since it didn't go into anything specific or interesting, I'm afraid it counts as shallow—especially because it's also a cliché response to NYer articles.

Haha - cliché or common knowledge?

Re: The computers are getting better at writing

#153

Post by dang »

nickpinkston wrote: post #152
dang wrote: post #145

Earlier quoted context omitted.

Since it didn't go into anything specific or interesting, I'm afraid it counts as shallow—especially because it's also a cliché response to NYer articles.

Haha - cliché or common knowledge?

From an HN moderation point of view I think those probably fall into the same bucket.

Re: The computers are getting better at writing

#154

Post by alvah »

NoOneNew wrote: post #114
bertil wrote: post #69

Earlier quoted context omitted.

If you want to echo the last paragraph of the article, the system also doesn’t know when to stop. Which leads to a simple improvement: teach the model when to stop, what sections to remove. There’s already great ML-based summary engines, and systems able to answer questions from descriptions.

Yea, but that misses the point to all the qualms against the idea that this is "good writing". I know the name of this eassy structure changes depending on region and generation, but 3.5 essay structures teaches you the basic principles of "good writing". What are you arguing, quickly overview your 3 points to prove it, next 3 paragraphs explain and support each point, then a conclusion paragraph. Super simple, basic…

Correct me if I'm wrong, but I suspect you misunderstand the purpose of all those recipes with ~2000 words of ramble followed by ~300 words of actual recipe. The 2000 words of fluff isn't for the reader, it's 100% for the search engine. It's required to provide enough room to crowbar in the required keywords without triggering an over-optimisation penalty. All in the name of getting it ranked in the SERPs, so people will actually read the ~300 words, or at least click on an ad or two. Unfortunately this is where search engine algorithms have taken us, for now at least.

Re: The computers are getting better at writing

#156

Post by gwern »

8bitsrule wrote: post #65

Couldn't help but notice that 'Xanadu' uses a lot of rhymes ... GPT apparently didn't notice that constraint. And then there's: "While all the stars that round her burn’d, Bow’d to the ground and based their fires. To the one ever-branching cloud" When 'base' is used as a past-tense verb, we expect a prepositional phrase ... before the next period. The preposition is never 'to'. Doesn't matter; the period stopped the…

> Couldn't help but notice that 'Xanadu' uses a lot of rhymes ... GPT apparently didn't notice that constraint.

Due to an obscure low-level technical decision (encoding text into BPEs), phonetics are erased from the GPT training data, and so therefore neither GPT-2 nor GPT-3 can learn to rhyme or make puns (they can only memorize relatively common pairs by bruteforce): https://www.gwern.net/GPT-3#bpes

I put a good deal of effort into testing this, including "Kubla Khan": https://www.gwern.net/GPT-3#samuel-taylor-coleridge Just doesn't work, and it never will until OA trains a model using some sort of better encoding which preserves phonetic information. (Character-based, unigram LM, or BPE dropout might all work, we don't know yet.)

Re: The computers are getting better at writing

#157

Post by thaumasiotes »

camillomiller wrote: post #131
YeGoblynQueenne wrote: post #117

Earlier quoted context omitted.

>> One clear example: attributive nouns. Specific examples, please? I'm curious :)

In a case where a noun is a modifier of the following word, Italians tend to prefer specificity , therefore falling for the “noun + of…” construct. A chicken soup bowl is easy to understand, but for an Italian that’s a “piatto di brodo di pollo”, so an Italian native would tend to feel like “a bowl of chicken soup” is the better choice. In this case there’s no real mistake (and for “bowl of” you might argue there’s a…

Just for reference, the default English constructions would be "bowl of chicken soup" and "Andy's bowl of chicken soup".

"Bowl of soup of chicken" is right out, but "chicken soup bowl" isn't much better. It's so anomalous that I might interpret it as meaning "a bowl for chicken soup" as opposed to "a bowl with chicken soup in it right now".

Re: The computers are getting better at writing

#158

Post by WesolyKubeczek »

jamesjyu wrote: post #144
WesolyKubeczek wrote: post #90

I have a suspicion that, since GPT-3 gorged its enormously swollen fuzzy digital belly on a lot of real prose, it might, when prompted, occasionally throw in a piece of the prose in question, or a whole original passage, because to its fitting function, it would be a suitable thing to do. But then again, aren’t we all doing the same, regurgitating bits and pieces that we heard or read before and liked? Don’t we all s…

Founder of Sudowrite [1] here. > However, if anything, GPT-3 is mostly showing how little meaning there is, but in how many words. We imbue words and symbols with meanings, and we strive to give our stories meanings that transcend the prose on paper. Yes, it's true that GPT-3 can spout non-sequitars. But overall, it does stay on topic quite well, as compared to previous iterations like GPT-2. But sometimes, you WANT…

> Yes, it's true that GPT-3 can spout non-sequitars. But overall, it does stay on topic quite well, as compared to previous iterations like GPT-2.

I've heard GPT-3 could produce better text than BuzzFeed writers. You could read it without even spotting a difference.

It doesn't so much imply GPT-3 is so good (it is good, no denying that) as it implies how amazingly content-free BuzzFeed is. And mind you, there are people working on BF, writing articles and whatnot, and making pretty penny off it. Even more people read the stuff.

Maybe we the humans actually need to dilute grains of actual useful information into gobs of fluff to be able to process it well, just like vitamins have different bioavailability depending on which food they are contained within. I'm fine with this conclusion too. It's just amusing to be aware of that.

Next thing we're going to learn is that a remote-working middle manager at some large company who's rather good at his job and even has awarded annual bonuses, is discovered to have been a GPT-3 (or maybe GPT-4) simulacrum all along, with his subordinates and bosses alike being none the wiser. Which is going to say tons about the standards of management at that company.

Re: The computers are getting better at writing

#159

Post by bambax »

jamesjyu wrote: post #146
bambax wrote: post #102

Many good and prolific writers of the past used ghost writers who prepared drafts and idea lists for them, in effect making their work more like that of editors (it is said Alexandre Dumas worked like this -- it's one of the only reason to explain his spectacular productivity). So if AI is capable of doing some prep work, why not? When writing fiction, my struggle is usually not in the writing itself (making sentence…

Founder of Sudowrite here. We are built on GPT-3 but we have a bunch of things we do on top that optimizes Sudowrite for the narrative fiction/non-fiction use case. For example, we have a description tool that, given a highlighted phrase, gives you options for telling phrases in 5 senses and metaphorical uses. Another tool we have analyzes a story premise and gives you possible twist and story turns, based on the gen…

Interesting! Thank you for answering.

How long is the waiting list to join Sudowrite beta?

Re: The computers are getting better at writing

#160

Post by jamesjyu »

bambax wrote: post #159
jamesjyu wrote: post #146

Earlier quoted context omitted.

Founder of Sudowrite here. We are built on GPT-3 but we have a bunch of things we do on top that optimizes Sudowrite for the narrative fiction/non-fiction use case. For example, we have a description tool that, given a highlighted phrase, gives you options for telling phrases in 5 senses and metaphorical uses. Another tool we have analyzes a story premise and gives you possible twist and story turns, based on the gen…

Interesting! Thank you for answering. How long is the waiting list to join Sudowrite beta?

It’s gotten quite large, but we plan on sending more invites out soon.

We are prioritizing folks who are working writers and have an immediate project they’re working on that could benefit from collaborating with an AI. If this sounds like you, sign up on the beta form and also send me a note about how you’ll use Sudowrite to jamesjacobyu@gmail.com

Post reply on HN