Using a Markov chain to generate readable nonsense with 20 lines of Python
1–10 of 77 posts
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#2>> Making the prefix shorter tends to produce less coherent prose; making it longer tends to reproduce the input text verbatim. For English text, using two words to select a third is a good compromise; it seems to recreate the flavor of the input while adding its own whimsical touch.
Next word prediction; vector databases:
Vector database: https://en.wikipedia.org/wiki/Vector_database
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#3This was posted 50 days ago - how does it reach first page again?
https://news.ycombinator.com/from?site=benhoyt.com
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#4This was posted 50 days ago - how does it reach first page again? https://news.ycombinator.com/from?site=benhoyt.com
[deleted]
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#5This is exactly the approach I used back in the late 80s; I published a Markov chain text generator on comp.sources.games that digested Usenet posts, built tables and used the previous two words to generate the next word. It was mildly amusing at the time. Someone resurrected it and put on GitHub, which I'm fine with.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#6Never forget a mate implemented a markov chainer in Lambda MOO, and named it “Cute Ickle Baby”. Left it in the Living Room, listening to what people said, and randomly saying things. Sometimes it said the funniest shit and had us all pissing ourselves laughing.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#7This works pretty well for languages that use a space as word separator. Now do this for those languages that don’t and have no clear and easy rules for word separation.
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#8This is exactly the approach I used back in the late 80s; I published a Markov chain text generator on comp.sources.games that digested Usenet posts, built tables and used the previous two words to generate the next word. It was mildly amusing at the time. Someone resurrected it and put on GitHub, which I'm fine with.
Mark V. Shaney?
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#9As many know and point out, this idea is now very old (40+ years?). The big problem is that it creates "word salad" [0]. In the generative art and procgen community, the term "10,000 bowls of oatmeal" problem has been used [1].
Taking a step back, this is a perennial problem, even with AI old and new. I've heard that the early (good) chess bots, after Deep Blue, had a problem of only being locally context sensitive and being easily fooled by longer range attack planning. We even see it, to a certain extent, in LLMs where they forget or contradict themselves with what was just said. I have no doubt that LLMs will just keep getting better but, as a snapshot of right now, you can see shades of this "word salad" problem, just a few steps removed to become a "logic salad" problem.
[0] https://en.wikipedia.org/wiki/Word_salad
[1] https://galaxykate0.tumblr.com/post/139774965871/so-you-want...
Re: Using a Markov chain to generate readable nonsense with 20 lines of Python
#10I wonder how this would behave if trained on 1 trillion words like the LLMs.
Also, Is training a Markov cheaper that training neural nets? It would be a great way to cut AI costs if they could be made as effective as neural nets.
There is also an interesting post by Orange duck. [0]