Live data from Hacker News

Twitter Bot Finds Anagrams of Twitter Statuses

anagramatron.tumblr.com

11–20 of 23 posts

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#11
post #2

"Mustache got thicker" vs "git checkout hamster" This bot is pretty funny. Anyone know how it works? I'm assuming it just sorts the string and puts it in a hashmap/table and looks for collisions.

I thought this was deep:

Go All out or Die trying.

v/s

R u really going to do it?

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#12
post #7

I'm a bit surprised that there are anagrams to be found. It's easy to find them if they exist, but there's no guarantee at all that there actually should be collisions.

I'm not a statistician. Is it really that surprising? English has plenty of redundancy; Twitter statuses have limited length. What's surprising to me is the niceness of the found anagrams. "another math genius" / "he ain't smart enough".

> What's surprising to me is the niceness of the found anagrams.

That's because they are manually curated [0]

   Q: Is this manually curated?

    A: Mostly for issues of volume ( there are a lot of variations 
    of 'goooood mooornnniinng!', there are a lot of spam bots 
    posting subtely different versions of the same message, etc) 
    the bot doesn't automatically post every anagram it finds. 
    Essentially there's an iphone client that reviews matches, 
    which are manually approved or rejected.
[0] https://github.com/cmyr/anagramatron

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#14

I'm a bit surprised that there are anagrams to be found. It's easy to find them if they exist, but there's no guarantee at all that there actually should be collisions.

It's actually extremely likely. The chance that any two statuses are anagrams is miniscule, and even the chance that a particular status has an anagram among all other statuses is probably small, but the chances that there are no collisions at all is tiny.

See a description of the Birthday Paradox[1] for the mathematics behind this. For example, if you put 70 people in a room, there is a 99.9% chance that two people share a Birthday.

[1]: http://en.wikipedia.org/wiki/Birthday_problem

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#15
post #2

"Mustache got thicker" vs "git checkout hamster" This bot is pretty funny. Anyone know how it works? I'm assuming it just sorts the string and puts it in a hashmap/table and looks for collisions.

Probably just takes tweets, canonicalizes them, and then hashes them based on a 26-length vector of character counts. For every new tweet, it looks for old tweets with the same character count.

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#16
post #7

Earlier quoted context omitted.

I'm not a statistician. Is it really that surprising? English has plenty of redundancy; Twitter statuses have limited length. What's surprising to me is the niceness of the found anagrams. "another math genius" / "he ain't smart enough".

I am a statistician. Maybe I should sit down and actually do some calculations.

Please do. I'd be interested.

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#18
I find it interesting that they're manually approving the hits, because, as they indicate, most hits are (nearly) identical.

It shouldn't be too difficult to solve this automatically though. Identical hits can be discarded very easily. The ones that only have a few words or letters reversed can be detected with some kind of similarity algorithm.

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#19
post #18

I find it interesting that they're manually approving the hits, because, as they indicate, most hits are (nearly) identical. It shouldn't be too difficult to solve this automatically though. Identical hits can be discarded very easily. The ones that only have a few words or letters reversed can be detected with some kind of similarity algorithm.

I had a look at the source code, and it does quite a bit of filtering, particularly around making sure the words are unique, and there is a primitive character comparison algorithm.

The code could be simplified by using Python's set() and improved by doing a copy'n'paste on a Levenshtein function.

Re: Twitter Bot Finds Anagrams of Twitter Statuses

#20

I'm a bit surprised that there are anagrams to be found. It's easy to find them if they exist, but there's no guarantee at all that there actually should be collisions.

Fermi estimate time!

Anagrams are just sentences with the same letter counts. The anagrams they're posting have 25ish letters... how many ways are there to distribute 25 balls into 26 bins? (25+26)!/25!/26! is ~250 trillion. The birthday paradox square roots that down to ~10 million, and the fact that we prefer some bins (fewer Zs, more Es) probably cuts it down even further to ~1 million.

So one anagram per million short tweets; hundreds per day. Doesn't seem too unreasonable.

Post reply on HN