Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I fed 24 years of my blog posts to a Markov model

susam.net

Post reply on HN
Print view View original

11–20 of 131 posts

Re: I fed 24 years of my blog posts to a Markov model

#11

Post by famouswaffles »

matusp wrote: post #9
atum47 wrote: post #3

I usually have this technical hypothetical discussions with ChatGpt, I can share if you like, me asking him this: aren't LLMs just huge Markov Chains?! And now I see your project... Funny

LLMs are indeed Markov chains. The breakthrough is that we are able to efficiently compute well performing probabilities for many states using ML.

LLMs are not Markov Chains unless you contort the meaning of a Markov Model State so much you could even include the human brain.

Re: I fed 24 years of my blog posts to a Markov model

#12

Post by roarcher »

atum47 wrote: post #3

I usually have this technical hypothetical discussions with ChatGpt, I can share if you like, me asking him this: aren't LLMs just huge Markov Chains?! And now I see your project... Funny

...are you under the impression that you have an exclusive relationship with "him"? Everyone else has access to ChatGPT too.

Re: I fed 24 years of my blog posts to a Markov model

#13

Post by cwyers »

matusp wrote: post #9
atum47 wrote: post #3

I usually have this technical hypothetical discussions with ChatGpt, I can share if you like, me asking him this: aren't LLMs just huge Markov Chains?! And now I see your project... Funny

LLMs are indeed Markov chains. The breakthrough is that we are able to efficiently compute well performing probabilities for many states using ML.

Yeah, there's only two differences between using Markov chains to predict words and LLMs:

* LLMs don't use Markov chains, * LLMs don't predict words.

Re: I fed 24 years of my blog posts to a Markov model

#14

Post by lloydatkinson »

pavel_lishin wrote: post #7
nurettin wrote: post #6

Earlier quoted context omitted.

Yes, I made one using bitlbee back in the 2000s, good times!

I made one for Hipchat at a company. I can't remember if it could emulate specific users, or just channels, but both were definitely on my roadmap at the time.

I'm hoping someone can find it so I can bookmark it but I once read a story about a company that let multiple markov chain bots loose in a Slack channel. A few days later production went down because one of them ran a Slack command that deployed or destroyed their infrastructure.

Re: I fed 24 years of my blog posts to a Markov model

#15

Post by bitwize »

vunderba wrote: post #5

I did something similar many years ago. I fed about half a million words (two decades of mostly fantasy and science fiction writing) into a Markov model that could generate text using a “gram slider” ranging from 2-grams to 5-grams. I used it as a kind of “dream well” whenever I wanted to draw some muse from the same deep spring. It felt like a spiritual successor to what I used to do as a kid: flipping to a random p…

Terry Davis, pbuh, did something very similar!

Re: I fed 24 years of my blog posts to a Markov model

#16

Post by bitwize »

0_____0 wrote: post #10
swyx wrote: post #2

now i wonder if you can compare vs feeding into a GPT style transformer of a similar Order of Magnitude in param count..

I thought for a moment your comment was the output of a Markov chain trained on HN

No mention of Rust or gut bacteria. Definitely not.

Re: I fed 24 years of my blog posts to a Markov model

#17

Post by sophrosyne42 »

famouswaffles wrote: post #11
matusp wrote: post #9

Earlier quoted context omitted.

LLMs are indeed Markov chains. The breakthrough is that we are able to efficiently compute well performing probabilities for many states using ML.

LLMs are not Markov Chains unless you contort the meaning of a Markov Model State so much you could even include the human brain.

Well LLMs aren't human brains, unless you contort the definition of matrix algebra so much you could even include them.

Re: I fed 24 years of my blog posts to a Markov model

#18

Post by anthk »

Megahal/Hailo (cpanm -n hailo for Perl users) can still be fun too.

Usage:

      hailo -t corpus.txt -b brain.brn
Where "corpus.txt" should be a file with one sentence per line. Easy to do under sed/awk/perl.

      hailo -b brain.brn
This spawns the chatbot with your trained brain.

By default Hailo chooses the easy engine. If you want something more "realistic", pick the advanced one mentioned at 'perldoc hailo' with the -e flag.

Re: I fed 24 years of my blog posts to a Markov model

#20

Post by hexnuts »

I just realized, one of the things that people might start doing is making a gamma model of their personality. I won't even approach who they were as a person, but it will give their Descendants (or bored researchers) a 60% approximation of who they were and their views. (60% is pulled from nowhere to justify my gamma designation, since there isn't a good scale for personality mirror quality for LLMs as far as I'm aware.)
Post reply on HN