Nice article! Bag-of-Words models, however, have their drawbacks and I wouldn‘t consider it a modern technique. Would be awesome if you would also try some state of the art techniques like NN word/document embeddings, e.g. with doc2vec.
The Federalist Papers: Author Identification Through K-Means Clustering
21–30 of 36 posts
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#22This is my first project in unsupervised NLP, so let me know if there's anything obviously wrong with the article or methodology.
nice article, there is a kaggle competition (Spooky Author) that you had to identify which of three authors wrote a sentence. the problem is very much the same, so not only can you try your technique on the data, you can also read kernels people posted in the competition.
Unlike what the commentor above said that its not modern and you should have did word2vec, bags of words are very robust and work well in these situations. word2vec was trained on a completely different corupus, and this data is quite small.
some things you might try are: - cosine distance between words - ad LDA (latent dirichelet allocation) topic probabilities - add verb speed (how fast they used the first verb in sentence - run an LSTM NN, add the predicted prob as features (careful in overfiting)
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#23This is my first project in unsupervised NLP, so let me know if there's anything obviously wrong with the article or methodology.
I know nothing about NLP but would a run where it predicts the authorship of the known texts be useful to get some idea of the level of accuracy?
Not the author, but I do know a bit about NLP, so to answer your question: Yes it would :)
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#24I did a similar exercise, and then applied it to the Titor posts. Turns out Alexander Hamilton is John Titor.
straight facts, my friend
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#25Now try Satoshi Nakamoto with bitcointalk posts and papers in cryptography.
Oh, he's James Madison.
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#26https://books.google.com/books/about/To_Couple_is_the_Custom...
1977. Always wondered how it stacked up against modern techniques
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#27This is a fun exercise. Back in 1963, Fred Mosteller and David L. Wallace wrote a piece in the Journal of the American Statistical Association titled "Inference in an Authorship Problem: A comparative study of discrimination methods applied to the authorship of the disputed Federalist papers" [0]. It describes another technique for analyzing the authorship using a Bayesian model of word distributions.
One interesting thing about this is the claim that there is a ground truth for all but 12 of the papers, meaning that supervised learning could also be used.
For discussion, I often think that unsupervised methods are preferred to supervised methods, given a reasonably low error rate by the unsupervised method, as it will be able to generalize more readily.
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#28Why not use something like LDA and features taken from word embeddings? A more probabilistic analysis would give more meaningful results.
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#29Now try Satoshi Nakamoto with bitcointalk posts and papers in cryptography.
Re: The Federalist Papers: Author Identification Through K-Means Clustering
#30Now try Satoshi Nakamoto with bitcointalk posts and papers in cryptography.
This assumes Satoshi discusses such things under a nym tied to his real name or IP.
I have a suite of nyms I created over Tor, and exclusively use via Tor, which I use to discuss certain topics.
It's hard to do a stylometric analysis when you don't have anything to compare to. Professional writing is very different from informal conversation.
If someone doesn't have a Facebook, gmail, twitter, etc it would be very difficult to find that person through stylometric analysis IMHO.