Live data from Hacker News

An algorithm for generating automatic hashtags

blog.swayy.co

11–20 of 35 posts

Re: An algorithm for generating automatic hashtags

#12

Although very basic, this is really nice - the wonders of NLTK! You explain your algorithm in very simple terms so kudos for that. It might be a good idea to put this in a GitHub repository rather than a simple gist maybe? I am sure there will be plenty of people (including myself) interested in contributing, it's much easier doing it on a repository rather than a gist :)

Good idea! Actually I'm thinking about creating some open source “social-NLP” python package. What do you think?

Re: An algorithm for generating automatic hashtags

#14
post #6

A strange example. Does the author really think hashtagifying the word 'content' has improved the tweet in some way? Do they expect people to be searching Twitter for #content and getting some useful results?

Probably not, but he did disclaim that it was pretty naive and could be improved in many ways. I think it's a pretty darn good first pass. That particular issues comes about from tagging words that are common in the target text without reference to whether or not that's actually significant- i.e., whether it's common in the text just because it's a common word overall, rather than because it's actually an indication…

Agree. Maybe some TF-IDF solution.

Re: An algorithm for generating automatic hashtags

#16
I think we - and to some degree the Twitter platform itself - are using hash tags redundantly, and this algorithm is just a manifestation of this redundancy that is killing data quality. These pathological tweets do tend to look like the example sentence provided, maybe even more extreme:

  #Swayy #Launches Into Public #Beta To Curate #Content For Your #SocialMedia Audience
Now, all of these words would be reachable with a normal search, so why do we over-tag everything? Are users really going to see what other Tweets have been recently tagged #Content? It makes even less sense with product names like #Swayy.

A more reasonable approach would be to tag things that are not part of the sentence itself:

  We're launching into public beta to curate content for social media! #Swayy
Or inline, on occasion, to express that you're taking part in a meme:

  Dear gods, #IHateIt when it's cold outside
We don't need algorithmic help to find hash tags in these cases either, and I'm arguing that automatically converting every third word into a hash tag doesn't do Twitter feeds any good, quality-wise.

Re: An algorithm for generating automatic hashtags

#17
post #9
post #6

A strange example. Does the author really think hashtagifying the word 'content' has improved the tweet in some way? Do they expect people to be searching Twitter for #content and getting some useful results?

https://twitter.com/search?q=%23Content

You can even get a #content t-shirt: http://bunnyaimee.tumblr.com/post/61678190440/we-opted-for-a...

Re: An algorithm for generating automatic hashtags

#20
The tech is nice, but adding hashtags like this does, in my opinion, more harm than good. Hashtagifying common words does rarely make sense, except in those few cases when there is a specific conversation going on about that word for any special reason.

Users ask us all the time to add an automatic hashtagging feature to hashtagify.me, but I'm resisting those requests because bad hashtagging makes hashtags less useful. It would be great to find an algorithm that (at least almost) always finds hashtags that are really relevant, but until that will happen it's better to ask users to make a little effort.

[edited for clarity]

Post reply on HN