I created a little program in C++/Qt for using Markov chains to re-write text. But it works on character sequences, rather than words. There are examples and binaries for Windows and Mac:
Using a Markov chain to generate readable nonsense with 20 lines of Python
61–70 of 77 posts
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#62As many know and point out, this idea is now very old (40+ years?). The big problem is that it creates "word salad" [0]. In the generative art and procgen community, the term "10,000 bowls of oatmeal" problem has been used [1]. Taking a step back, this is a perennial problem, even with AI old and new. I've heard that the early (good) chess bots, after Deep Blue, had a problem of only being locally context sensitive a…
i put Alexis De Tocqueville through this and the results were legible but hard to read. Then I put a demagogue politician speech through this and the results were almost like a speech that politician would have given. It was hard to tell the difference between the original and the generated. in other words, if the public admires word salad, and someone speaks in word salad, then word salad output would not be conside…
I did a markov reworking of some of Trump's speeches and got:
"Hillary brought death and disaster to Iraq, Syria and Libya, she empowered Iran, and she unleashed ISIS. Now she wants to raise your taxes very substantially. Highest taxed nation in the world is a tenant of mine in Manhattan, so many great people. These are people that have been stolen, stolen by either very stupid politicians ask me the question, how are you going to get rid of all the emails?” “Yes, ma’am, they’re gonna stay in this country blind. My contract with the American voter begins with a plan to end government that will not protect its people is a government corruption at the State Department of Justice is trying as hard as they can to protect religious liberty"
Details at: https://successfulsoftware.net/2019/04/02/bloviate/
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#63We had a bot that would randomly say things in our IRC channel 15 years ago that worked like this. You could also mention it to prompt it to reply. Every message was added to it's knowledge base and it would say random but hilarious stuff made up from all the nonsense we used to talk about. Good times.
Out of curiosity, was the bot based on the bMotion repository? https://github.com/jamesoff/bmotion I remember a friend of mine settings up an IRC bot (named Zeta) like that for his sheet music forum many years ago. She was involved in a lot of hilarity - probably my favorite antics were when she randomly decided to courtmatial someone. Good times indeed! :)
> she randomly decided to courtmatial someone
Well, you can't stave off fate - you've got to face the music eventually.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#64Earlier quoted context omitted.
i put Alexis De Tocqueville through this and the results were legible but hard to read. Then I put a demagogue politician speech through this and the results were almost like a speech that politician would have given. It was hard to tell the difference between the original and the generated. in other words, if the public admires word salad, and someone speaks in word salad, then word salad output would not be conside…
I recall one site way back that used a Markov chain generator to mash up Karl Marx and Ayn Rand. Was fairly plausible reading, actually.
I imagine that's like mixing different kinds of salad.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#65Earlier quoted context omitted.
i put Alexis De Tocqueville through this and the results were legible but hard to read. Then I put a demagogue politician speech through this and the results were almost like a speech that politician would have given. It was hard to tell the difference between the original and the generated. in other words, if the public admires word salad, and someone speaks in word salad, then word salad output would not be conside…
I did a markov reworking of some of Trump's speeches and got: "Hillary brought death and disaster to Iraq, Syria and Libya, she empowered Iran, and she unleashed ISIS. Now she wants to raise your taxes very substantially. Highest taxed nation in the world is a tenant of mine in Manhattan, so many great people. These are people that have been stolen, stolen by either very stupid politicians ask me the question, how ar…
When ngrams is high (10) how do you pick "what" to start from?
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#66Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#67Earlier quoted context omitted.
I did a markov reworking of some of Trump's speeches and got: "Hillary brought death and disaster to Iraq, Syria and Libya, she empowered Iran, and she unleashed ISIS. Now she wants to raise your taxes very substantially. Highest taxed nation in the world is a tenant of mine in Manhattan, so many great people. These are people that have been stolen, stolen by either very stupid politicians ask me the question, how ar…
When ngrams is high (10) how do you pick "what" to start from?
IIRC it starts with the same few characters as the source text.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#68Plausible nonsense of sufficient complexity and consistency as to seem real. It's trivial to generate.
It casts every cherished truth into doubt.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#69See also Mark V Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney This was a class assignment in college, we had a lot of fun with it, and one of my classmates, the brilliant Alyosha Efros, decided to apply the exact same technique to images instead of text. It turned into a paper that revitalized texture synthesis in the Siggraph community. The most interesting part about it (in my opinion) is that he ran it on im…
That's amazing! It's a literal uncrop. I'll have to look into this later.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#70As many know and point out, this idea is now very old (40+ years?). The big problem is that it creates "word salad" [0]. In the generative art and procgen community, the term "10,000 bowls of oatmeal" problem has been used [1]. Taking a step back, this is a perennial problem, even with AI old and new. I've heard that the early (good) chess bots, after Deep Blue, had a problem of only being locally context sensitive a…
The output is "word salad" because the probabilities of the model are uniform, albeit implicitly- eyballing the python, it selects the next n-gram uniformly at random. A better n-gram model would calculate the probability of an n-gram following another n-gram according to their frequency in the training corpus. With a larger corpus and better training techniques (some smoothing for one thing) you'd get much more cohe…
> absolutely no progress towards a "cognitive model of human grammatical knowledge"
The thing is, it's only if you buy fully into Chomskyan thinking that you think a "cognitive model of human grammatical knowledge" might even be useful. Or that it's a particularly special thing compared to any other cognitive model of human knowledge.