Live data from Hacker News

A kinda okay text generator

blog.ftql.pw

11–20 of 33 posts

Re: A kinda okay text generator

#11
post #8

What does "cweep" mean?

If it helps, here's the full paragraph:

  “We’ll send the infantwy down by the swamps,” Denísov continued.
  “They’ll cweep up to the garden; you’ll wide up fwom there with the
  Cossacks”—he pointed to a spot in the forest beyond the village—“and I
  with my hussars fwom here. And at the signal shot...”

Re: A kinda okay text generator

#13
post #7

I feel for fooling a quick skim, getting interpunction/capitalization correct/believable was the biggest improvement.

Ha.. yeah, I left that bit out. It was several dead ends and I finally ended up with something that was stupid simple. It seemed like more trouble to explain than it was worth.

Re: A kinda okay text generator

#15

One problem I see in the article above is that the result is evaluated only by eyballing- which is actually a perfectly legitimate evaluation method; humans' language faculty, is, after all, the only process we know of that can correctly recognise and generate natural language expressions (and other, automatic, methods like BLEU scores, often simply automate the eyballing process). However, if you're working with sta…

> they seem to have dipped a toe into statistical language modelling but be unwilling to go all the way with more advanced approaches

That's fair.

Thanks for your suggestions. I clearly have some studying to do.

Re: A kinda okay text generator

#16
A nice experiment! I am always a bit surprised to see that CNNs & company produce much less coherent and natural sounding text than the comparably simple Markov Chain generators.

That said, I markov-generated some text with Nietzsche's "Also sprach Zarathustra" as a source some time ago and it made more sense and was more coherent than the original.

Re: A kinda okay text generator

#17

A nice experiment! I am always a bit surprised to see that CNNs & company produce much less coherent and natural sounding text than the comparably simple Markov Chain generators. That said, I markov-generated some text with Nietzsche's "Also sprach Zarathustra" as a source some time ago and it made more sense and was more coherent than the original.

I don't think this is necessarily true. CNNs can produce incredible results, see for example [0]. A snippet from the paper (their code is also available):

""" Prompt: The Mage, the Warrior, and the Priest

Story: A light breeze swept the ground, and carried with it still the distant scents of dust and time-worn stone. The Warrior led the way, heaving her mass of armour and muscle over the uneven terrain. She soon crested the last of the low embankments, which still bore the unmistakable fingerprints of haste and fear. She lifted herself up onto the top the rise, and looked out at the scene before her. [...] """

RNN/LSTM type models can also produce really coherent results on the scales shown in the blog.

Markov chains can be quite useful when you can manually create conditioning, but some of the key power of neural methods is learning to process conditioning and blend in ways that may not be obvious (or require significant domain expertise to grok) when handcrafting. YeGoblynQueenne below discusses a lot of the core issues for Markov chain based approaches.

In particular, plagiaristic sequences are incredibly common in Markovian generation, and though there are some papers on how to deal with this [1] it is not straightforward to decide what plagiarism really means in many contexts. This problem also arises in neural models, but isn't nearly as extreme due to the nature of both learning and the sampling process.

One huge bonus of Markov chains is that controlling exactly what you want or using hard rules is pretty easy, which is definitely NOT the case with neural generation...

This blog post is a really nice run through of how to get decent results from Markov chains, but don't discount neural methods either - they are getting better all the time, if you are willing to deal with the headaches.

[0] Hierarchical Neural Story Generation, https://arxiv.org/abs/1805.04833

[1] Max Order http://www.flow-machines.com/maxorder/

Re: A kinda okay text generator

#18
post #11
post #8

What does "cweep" mean?

If it helps, here's the full paragraph: “We’ll send the infantwy down by the swamps,” Denísov continued. “They’ll cweep up to the garden; you’ll wide up fwom there with the Cossacks”—he pointed to a spot in the forest beyond the village—“and I with my hussars fwom here. And at the signal shot...”

Wewease Bwian!

Re: A kinda okay text generator

#19
post #15

One problem I see in the article above is that the result is evaluated only by eyballing- which is actually a perfectly legitimate evaluation method; humans' language faculty, is, after all, the only process we know of that can correctly recognise and generate natural language expressions (and other, automatic, methods like BLEU scores, often simply automate the eyballing process). However, if you're working with sta…

> they seem to have dipped a toe into statistical language modelling but be unwilling to go all the way with more advanced approaches That's fair. Thanks for your suggestions. I clearly have some studying to do.

[deleted]

Re: A kinda okay text generator

#20
Children's poetry makes for fun source texts for these projects. In this case dumping a bunch of old poems generated mostly gibberish but I found the following examples amusing.

While here on my deathbed I try to relate My many misfortunes and miseries great. Poor thoughtless young thing! If I recollect right, I began life in March, on a clear frosty night; And before I could see or was half a week old, I nearly had perished, the barn was so cold. But this chilly spring I got pretty well over, But there wasn't a tree for miles around, They were too frightened to stay on the ground, And moused in the stable, or played in the clover, Or till I was weary, which seldom occurred, Ran after my tail, which I took for a bird

The wind did blow, the cloak did fly, Like streamer long and gay, Till loop and button failing both, At last it flew away. Then might all people well discern The bottles he had slung, A bottle swinging at each side, As hath been said or sung. The dogs did bark, the children screamed, Up flew the windows all, And ev'ry soul cried out, Well done!

When Betty screaming came down stairs, The wine is left behind! Good lack! Quoth he yet bring it me, My leathern belt likewise, In which I bear my trusty sword When I do exercise. Now Mistress Gilpin, careful soul!

There as the mother sits all day, On business from their houses, And late at night returning home, To cheer their babes and spouses; While you and I have oft-times heard How men are killed and undone, By overturns from carriages, By thieves, and fires in London. We know what risks these landsmen run, From noblemen to tailors.

Post reply on HN