Live data from Hacker News

Using Machine Learning and Node.js to detect the gender of Instagram Users

totems.co

1–10 of 56 posts

Re: Using Machine Learning and Node.js to detect the gender of Instagram Users

#2
It's unusual to see a coherent, from-first-principles explanation of a neural network. Especially one that's commercially valuable (i presume) to Totems.

Mildly alarmed to learn I'm only .039 probability male, though - better bloke it up on Instagram.

Re: Using Machine Learning and Node.js to detect the gender of Instagram Users

#3
> Our platform retrieves or refreshes around 400 user profiles per second (this is managed using 4 high-bandwidth servers co-located with instagram’s API servers on AWS).

Interesting, since Instagram's API only allows 5,000 requests per hour, (http://instagram.com/developer/limits/) and does not support bulk requests of user data. How does this application bypass this limit?

Re: Using Machine Learning and Node.js to detect the gender of Instagram Users

#4

> Our platform retrieves or refreshes around 400 user profiles per second (this is managed using 4 high-bandwidth servers co-located with instagram’s API servers on AWS). Interesting, since Instagram's API only allows 5,000 requests per hour, ( http://instagram.com/developer/limits/ ) and does not support bulk requests of user data. How does this application bypass this limit?

Hi minimaxir. We have a large number of tokens from our clients and people doing oauth to access our free demo. Since the data is public, we can use any of these tokens to access hashtags and account followers, etc...

Actually, Instagram API limit is pretty high when compared to other platform. Today we have something like 100k tokens available to us, which means we can make 12bn+ calls everyday. Almost like having a firehose. We don't use all of them but we're one of the top users. Though there are at least 10 bigger users than us on the API (according to them of course). Hope it helps!

Re: Using Machine Learning and Node.js to detect the gender of Instagram Users

#5

> Our platform retrieves or refreshes around 400 user profiles per second (this is managed using 4 high-bandwidth servers co-located with instagram’s API servers on AWS). Interesting, since Instagram's API only allows 5,000 requests per hour, ( http://instagram.com/developer/limits/ ) and does not support bulk requests of user data. How does this application bypass this limit?

You only need to use 288 tokens to hit 400 user profiles/second.

Re: Using Machine Learning and Node.js to detect the gender of Instagram Users

#7
post #4

> Our platform retrieves or refreshes around 400 user profiles per second (this is managed using 4 high-bandwidth servers co-located with instagram’s API servers on AWS). Interesting, since Instagram's API only allows 5,000 requests per hour, ( http://instagram.com/developer/limits/ ) and does not support bulk requests of user data. How does this application bypass this limit?

Hi minimaxir. We have a large number of tokens from our clients and people doing oauth to access our free demo. Since the data is public, we can use any of these tokens to access hashtags and account followers, etc... Actually, Instagram API limit is pretty high when compared to other platform. Today we have something like 100k tokens available to us, which means we can make 12bn+ calls everyday. Almost like having a…

> Since the data is public, we can use any of these tokens to access hashtags and account followers

I'm fairly certain this isn't an intended use of the API. Kudos to you for posting your method here, but I wouldn't be surprised if your app gets banned because of this (a lot of people at Instagram read HN).

Re: Using Machine Learning and Node.js to detect the gender of Instagram Users

#8
post #4

> Our platform retrieves or refreshes around 400 user profiles per second (this is managed using 4 high-bandwidth servers co-located with instagram’s API servers on AWS). Interesting, since Instagram's API only allows 5,000 requests per hour, ( http://instagram.com/developer/limits/ ) and does not support bulk requests of user data. How does this application bypass this limit?

Hi minimaxir. We have a large number of tokens from our clients and people doing oauth to access our free demo. Since the data is public, we can use any of these tokens to access hashtags and account followers, etc... Actually, Instagram API limit is pretty high when compared to other platform. Today we have something like 100k tokens available to us, which means we can make 12bn+ calls everyday. Almost like having a…

@yid. I'd be curious to understand why you think they would ban us? For using the tokens this way? Well they know everything about our usage of these tokens, and any analytics tool out there behave similarly, right?

Re: Using Machine Learning and Node.js to detect the gender of Instagram Users

#9

1.000 probability of being a man. Thank you for affirming my masculinity. However, my business has a 0.885 probability of being a woman, which is odd for a men's brand.

Hehe, the network was trained mostly on humans (see the data set section)... So all bets are off for a non human account :)
Post reply on HN