Live data from Hacker News

The computers are getting better at writing

newyorker.com

141–150 of 165 posts

Re: The computers are getting better at writing

#141

Earlier quoted context omitted.

The outputs on pageload are just a random uncurated set I ran. I never claimed smaller models like the MtG models were immune to the same curation issues as larger text generation models. As someone who did try to reproduce RoboRosewater with LSTMs years ago, I can say using GPT-2 overall has a higher signal-to-noise ratio in terms of generating interesting cards which is a win.

>> The outputs on pageload are just a random uncurated set I ran. I never claimed smaller models like the MtG models were immune to the same curation issues as larger text generation models. Are you sure? I reloaded the page once, before commenting above, but the output didn't change. I keep reloading now and the same set of cards is displayed. I reloaded with Shift-F5 to be sure ish. >> As someone who did try to rep…

You have to run the notebook itself by following the instructions at the top, then you can customize the output (lower temperatures will follow the rules better).

Here's some examples of curated cards generated that probably fit your description of "interesting": https://www.reddit.com/r/magicTCG/comments/n396g8/i_trained_...

Re: The computers are getting better at writing

#142
post #48

Rewinding a bit from computers doing all the writing, are you using any tools that help you write considerably better (not just spelling/punctuation checking)? As a non-native English speaker I would like to have a tool that points out my nonstandard usages of language. I think it could be something as simple as parsing the text and searching it against a corpus, but I have no idea how many "unique structures" a text…

As a fellow non-native who accepted not to strive for full assimilation, my suggestion is to do the same. You can refine your writing by reading a lot. Books like “The elements of style” and “On writing well” can help with grasping those structural elements you might be struggling to identify. Apart from that, our brain has been shaped by our native language, and we can’t really delete that. Which is great news! It m…

And English, in its predatory way, will adopt those 'different' words and constructions as its own in due time :)

Re: The computers are getting better at writing

#143

The lesson we should take from GPT-3 is that we spend way too much time reading text that isn't really saying anything, and/or failing to find meaning in what we read. It's a neat trick, and I think it could absolutely replace much or most of the writing on the internet today. But is it good writing? No, not really. At least, not in the sense of writing as producing anything more substantive than slick prose with a v…

Founder of Sudowrite [1] here.

> I think the next step (say, being able to produce short stories with actual literary value) will be substantially more difficult, mostly because it's substantially more difficult for human writers too.

Totally agreed. Both my co-founder and I are professionally published short story authors, and we would be the first to say that using something like Sudowrite is no guarantee you will land a story in the New Yorker. Similarly, using Cinema4D won't propel you to the same quality work as Beeple. These are tools. It's up to you as the author to use them in a way that enhances your craft.

My goal is to figure out how we mindfully incorporate AI into a writer's workflow. If you look at other fields like visual arts, they have had a wealth of interactive tools starting with photoshop. What is the photoshop for writing? (hint: I don't believe it exists yet, and I want to build it)

In terms of the details, you'd be surprised at how well GPT-3 handles descriptions and dialogue. It's actually scarily good at dialogue [2] [3].

[1] https://www.sudowrite.com

[2] https://www.gwern.net/GPT-3#dialogue

[3] https://www.jamesyu.org/singular/

Re: The computers are getting better at writing

#144

I have a suspicion that, since GPT-3 gorged its enormously swollen fuzzy digital belly on a lot of real prose, it might, when prompted, occasionally throw in a piece of the prose in question, or a whole original passage, because to its fitting function, it would be a suitable thing to do. But then again, aren’t we all doing the same, regurgitating bits and pieces that we heard or read before and liked? Don’t we all s…

Founder of Sudowrite [1] here.

> However, if anything, GPT-3 is mostly showing how little meaning there is, but in how many words.

We imbue words and symbols with meanings, and we strive to give our stories meanings that transcend the prose on paper. Yes, it's true that GPT-3 can spout non-sequitars. But overall, it does stay on topic quite well, as compared to previous iterations like GPT-2.

But sometimes, you WANT that, you want to be provoked to get unstuck on writing that tough scene or character. I've found that using a system like Sudowrite is not only about generating new text, but reading the generations and discovering how the neural net is interpreting what you've written. I believe this particular use case hasn't been fully explored, outside of summarization algorithms.

[1] https://www.sudowrite.com/

Re: The computers are getting better at writing

#145
post #43

Earlier quoted context omitted.

" Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html

I wouldn't say it's shallow. I think the New Yorker's editorial board / style guide does a disservice in explaining science / engineering concepts. I just delivered my critique via sarcasm.

Since it didn't go into anything specific or interesting, I'm afraid it counts as shallow—especially because it's also a cliché response to NYer articles.

Re: The computers are getting better at writing

#146
post #102

Many good and prolific writers of the past used ghost writers who prepared drafts and idea lists for them, in effect making their work more like that of editors (it is said Alexandre Dumas worked like this -- it's one of the only reason to explain his spectacular productivity). So if AI is capable of doing some prep work, why not? When writing fiction, my struggle is usually not in the writing itself (making sentence…

Founder of Sudowrite here.

We are built on GPT-3 but we have a bunch of things we do on top that optimizes Sudowrite for the narrative fiction/non-fiction use case.

For example, we have a description tool that, given a highlighted phrase, gives you options for telling phrases in 5 senses and metaphorical uses. Another tool we have analyzes a story premise and gives you possible twist and story turns, based on the genre of your story, and also gives you possible characters that could inhabit that world, complete with both external and internal character descriptions.

I agree that for writing stories, the hard part is in the forest and not the trees (although this isn't true for everyone). And systems like GPT-3 can help with that, too, simply because it's level understanding of text is approaching that of a human.

Re: The computers are getting better at writing

#147

Earlier quoted context omitted.

I expect GPT-3 to be amazing for people who want to write stories or articles but are really terrible at one specific aspect of the writing process.

https://slideplayer.com/slide/6081467/18/images/12/The+scale... It depends on if you think writing ability is more like the top graph, or the bottom graph. If it's like the top graph, humans have plenty of time to adopt AI tools, and incorporate them into the writing process. If it's like the bottom graph, then GPT-4 comes out in 12 months and is human equivalent, and GPT-5 comes out a year after that and is a better…

Progress so far is that GPT-1 could produce a coherent sentence but stopped making sense after a few lines. GPT-2 could produce a coherent paragraph, but several paragraphs strung together made no sense. GPT-3 is coherent for about a page or screen, but not a chapter.

A page is good enough for the clickbait industry.

Re: The computers are getting better at writing

#148

Earlier quoted context omitted.

>> The outputs on pageload are just a random uncurated set I ran. I never claimed smaller models like the MtG models were immune to the same curation issues as larger text generation models. Are you sure? I reloaded the page once, before commenting above, but the output didn't change. I keep reloading now and the same set of cards is displayed. I reloaded with Shift-F5 to be sure ish. >> As someone who did try to rep…

You have to run the notebook itself by following the instructions at the top, then you can customize the output (lower temperatures will follow the rules better). Here's some examples of curated cards generated that probably fit your description of "interesting": https://www.reddit.com/r/magicTCG/comments/n396g8/i_trained_...

Sorry, you said on pageload, I thought you meant when the page is loaded :)

On the cards you posted to reddit:

a) "Keening Vythat", "Tombsis Satyr", "Quarpling Barrier". These are typical nonsense-nonsense names (as opposed to the fantasy-nonsense names found on M:tG cards) that are typical, in my experience, of card names generated by language models trained at the character level. M:tG fantasy-nonsense names, like "Rathi", "Kapashen", "Viashino", etc, are derived from a real-world narrative created specifically to theme the cards. A backstory. Without such a backstory, "Vythat", "Tombsis", "Quarpling" are just random strings that have no reason to be used as names. This is an important limitation of this kind of card generation, especially given the very low chance that you'll see any more "Vythat" cards (and if you did, they'd likely have a completely different function). Also "Keening Vythat" sounds like a creature, rather than a Sorcery.

So, first problem: thematic consistency of names is all over the place.

b) The text on Keening Vythat is ungrammatical:

  Basic lands are basic land type in addition to its other types.
c) Seismic Mind has an interesting ability (the one that modifies its base P/T). The mana cost makes sense, actually (although I bet it'd end up at {4U} at R&D), and the effect is very Blue. Nice.

d) Tombsis Satyr also has an even more interesting ability (feeds the inner Johnny). The cost and colours are right and the Angel type is just the cherry on top. Very nice. If only it wasn't called "Tombsis Satyr"... X)

e) Quarpling Barrier's ability is more of a crippling weakness than an interesting ability. A creature without Defender would not be called a "Barrier" these days, either. Pass, I fear.

f) Trade Damage is more balanced than interesting, but it's very well balanced! It would fit very nicely in a modern tempo deck. Nice also.

So that's three nice, two not so nice and a bit of a problem with themes. Still, those are not that bad at all. Unfortunately, there's just five of them. At this point it's hard to tell whether your project is able to generate better cards than other projects - and why would it, if it's using the same tech as those projects? You can fine tune and turn more knobs, but what, fundamentally, is the innovation here?

I suggest that you add the five cards above (and/or others) to the colab of your project alongside the auto-generated ones, to have examples of the "best case" and "average case" together. That'll give a boost to the appeal of your project.

Anyway, better than I originally thought but still a ways to go.

Re: The computers are getting better at writing

#149

Earlier quoted context omitted.

You have to run the notebook itself by following the instructions at the top, then you can customize the output (lower temperatures will follow the rules better). Here's some examples of curated cards generated that probably fit your description of "interesting": https://www.reddit.com/r/magicTCG/comments/n396g8/i_trained_...

Sorry, you said on pageload, I thought you meant when the page is loaded :) On the cards you posted to reddit: a) "Keening Vythat", "Tombsis Satyr", "Quarpling Barrier". These are typical nonsense-nonsense names (as opposed to the fantasy-nonsense names found on M:tG cards) that are typical, in my experience, of card names generated by language models trained at the character level. M:tG fantasy-nonsense names, like…

Again, this (and my other text generation projects) are optimizing more for humor/chaos than accuracy, and there is a bolded important notice before the text emphasizing that the results may not be legal. Per the other comments on that Reddit thread, that appears to be the correct approach as the crazier-but-barely-legal cards are what resonate the best (the "bad" RoboRosewater cards were the most popular, not the good ones).

Curating only cards which are grammatically good is in itself misleading. Part of the reason I work on text generation in the first place is to illustrate its limitations.

I encourage you to run the notebook itself instead of arguing a No True Scotsman about something I never tried to assert. This notebook is not a thesis paper, just a side project about a ML model that took more time to train than to write the code for.

Re: The computers are getting better at writing

#150
post #57

Earlier quoted context omitted.

prowritingaid has a very good, free, web-based tool that will parse out all sorts of neat things in your work. for example: i write fiction and have a real problem with “echos” — words that i repeat within a certain distance of each other — and i even miss them when proofing/editing. pwa is a godsend for this. if scrivener would add their own native version of this (hint hint incase any of you L&L people lurk here..…

Well, repetition is considered harmful in literary writing but that, too, is just a convention. There's no reason to force yourself not to do it if you find that it comes to you naturally. Rather, I'd try to find a way to develop it further into a unique style instead. Think of minimal music, heavy metal riffs and chorus...es (chori?), or repeating leitmotifs in classical music. Repetition drives a point home. It's a…

i’m with you on the technical bit. in any style of writing where the goal is to concisely convey information, you’re definitely right that style doesn’t matter in most cases, so repeating words isn’t really something to avoid. to kinda piggyback off your anecdote, a lot of people trip into that pitfall of trying to make whatever they’re writing sound too unique, and in doing so, they (often inadvertently) add a layer of verbosity that adds wordiness instead of anything meaningful to what they’re trying to say.

my issue is that i reuse adverbs and adjectives within a few sentences of one anther and simply don’t notice it — like saying “rain fell in gentle waves,” then a sentence or two later, “the tide rolled in, soft, gentle, crashing against…” it just feels.. lazy, unimaginative to me. i read somewhere that it’s a quirk of the brain (perhaps only for some people, i don’t remember) where words in immediate memory get looped and “stuck” there. that seems to be what happens when i write, and it chaps my ass.

Post reply on HN