Generating random text
cs.bell-labs.com
Generating random text
1–10 of 29 posts
Re: Generating random text
#2I remember reading about that in "The Information", where it is described how Claude Shannon did it.
Now I'll have to tweak my spam detection even more. Joke aside and somebody correct me if I'm wrong, spam probably runs on a simple wordlist type algorithm.
What is then the usefulness (I define usefulness extremely wide) of such a generator?
Re: Generating random text
#3Re: Generating random text
#4I remember reading about that in "The Information", where it is described how Claude Shannon did it. Now I'll have to tweak my spam detection even more. Joke aside and somebody correct me if I'm wrong, spam probably runs on a simple wordlist type algorithm. What is then the usefulness (I define usefulness extremely wide) of such a generator?
If you have a good generator for text, you have a useful language model that can be plugged into applications such as speech recognition, OCR, predictive text entry systems and compression.
Re: Generating random text
#5Markov chains are always good fun to play with... a few months ago I worked on a class project which generated markov models from Final Fantasy SNES tracks. (https://github.com/IsaacLewis/MidiMarkov). I should blog about it at some point.
I hadn't seen Shannon's algorithm before though, which looks a bit more memory efficient than the approach I used.
Re: Generating random text
#6Another practical example of Markov-chaining is http://www.x11r5.com/ - a robot that's various trained from IRC, Twitter and Identica content.
There's even a weekly podcast generated from news headlines: http://www.x11r5.com/radio/
Re: Generating random text
#7I remember reading about that in "The Information", where it is described how Claude Shannon did it. Now I'll have to tweak my spam detection even more. Joke aside and somebody correct me if I'm wrong, spam probably runs on a simple wordlist type algorithm. What is then the usefulness (I define usefulness extremely wide) of such a generator?
If you have a good generator for text, you have a useful language model that can be plugged into applications such as speech recognition, OCR, predictive text entry systems and compression.
Could you somehow use it in reverse? What I mean is, is it possible to get a random text generator for a certain language and then use it to determine, whether a given text is in that language or not?
Re: Generating random text
#8Earlier quoted context omitted.
If you have a good generator for text, you have a useful language model that can be plugged into applications such as speech recognition, OCR, predictive text entry systems and compression.
Could you somehow use it in reverse? What I mean is, is it possible to get a random text generator for a certain language and then use it to determine, whether a given text is in that language or not?
Yes.
Given the string so far, see how probable it is that the generator would generate the next character, p(x_n | x_For more on information theory, modelling and inference you might like: http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
Re: Generating random text
#9"Dive into Python" contains chapter on XML processing[1] that uses grammar defined in XML to generate random English passages. A very funny and informative reading.
Re: Generating random text
#10Earlier quoted context omitted.
If you have a good generator for text, you have a useful language model that can be plugged into applications such as speech recognition, OCR, predictive text entry systems and compression.
Could you somehow use it in reverse? What I mean is, is it possible to get a random text generator for a certain language and then use it to determine, whether a given text is in that language or not?
It sounds like you are talking about a naive Bayesian classifier. PG wrote a couple of articles on his experience with these for spam filtering (http://www.paulgraham.com/spam.html and http://www.paulgraham.com/better.html). They're probably a decent high-level introduction to the area.
For a more in-depth, yet very accessible discussion, I would recommend "Speech and Language Processing" by Jurafsky & Martin (http://books.google.com/books/about/SPEECH_AND_LANGUAGE_PROC...). It's considered by many to be the Bible of NLP.