Markov Chain Text Generation
blog.demofox.org
Markov Chain Text Generation
1–10 of 23 posts
Re: Markov Chain Text Generation
#2You can use Google search results as a Markov chain (with the ranks translated to probabilities). Then your chain is fitted to all of the visible web.
Another cool use for ML is for categorical and textual data. A fitted Markov Chain can give the probability of a string occuring. Strings that are more random (such as spammy text) get a low probability. Strings that are similar (such as certain user agents) get a similar probability, without having to directly compare these.
Re: Markov Chain Text Generation
#3Years ago I used Twitter streams as input for markov chain text generation. It’s interesting to recognize your own writing style in what is essentially gibberish: http://yes.thatcan.be/my/next/tweet
Re: Markov Chain Text Generation
#4Ask HN: Markov Chain book recommendations?
Re: Markov Chain Text Generation
#5Considering RNNs have completely taken over text generation with language models, are there any papers that comprehensively compare the power of RNN and Markov models?
Re: Markov Chain Text Generation
#6Considering RNNs have completely taken over text generation with language models, are there any papers that comprehensively compare the power of RNN and Markov models?
In a sense, they're equivalent. RNN's create models of probabilistic state transitions. In other words, they create Markov chain models. You might argue that a Markov chain doesn't have memory, but it's easy to incorporate "memory" by expanding the definition of a "state" to include recent history or any function of the past.
Re: Markov Chain Text Generation
#7Ask HN: Markov Chain book recommendations?
Bayesian reasoning and machine learning by Barber (2012) had nice introductory texts on most of the mechanisms at work iirc if you're looking for a lecture oriented resource.
Re: Markov Chain Text Generation
#8Considering RNNs have completely taken over text generation with language models, are there any papers that comprehensively compare the power of RNN and Markov models?
In a sense, they're equivalent. RNN's create models of probabilistic state transitions. In other words, they create Markov chain models. You might argue that a Markov chain doesn't have memory, but it's easy to incorporate "memory" by expanding the definition of a "state" to include recent history or any function of the past.
No, they're not. And if it was "that easy" to incorporate memory RNNs would not have replaced Markov Chains
As an example try to build a character level Markov chain for text generation and see how that goes.
Re: Markov Chain Text Generation
#9Earlier quoted context omitted.
In a sense, they're equivalent. RNN's create models of probabilistic state transitions. In other words, they create Markov chain models. You might argue that a Markov chain doesn't have memory, but it's easy to incorporate "memory" by expanding the definition of a "state" to include recent history or any function of the past.
No, they're not. And if it was "that easy" to incorporate memory RNNs would not have replaced Markov Chains As an example try to build a character level Markov chain for text generation and see how that goes.
The point is that an RNN or other language model describes a stochastic process which can be seen as a Markov process with state space given by the internal state of the RNN. This doesn't say anything about the ease of implementing or learning such a model. Just that from a purely mathematical perspective, they are equivalent in power.
Re: Markov Chain Text Generation
#10Earlier quoted context omitted.
No, they're not. And if it was "that easy" to incorporate memory RNNs would not have replaced Markov Chains As an example try to build a character level Markov chain for text generation and see how that goes.
The point is that an RNN or other language model describes a stochastic process which can be seen as a Markov process with state space given by the internal state of the RNN. This doesn't say anything about the ease of implementing or learning such a model. Just that from a purely mathematical perspective, they are equivalent in power.
Ah I see, I agree with their equivalence in the mathematical sense.