Live data from Hacker News

Show HN: Ghuser.io – Complete GitHub profiles

github.com

11–20 of 52 posts

Re: Show HN: Ghuser.io – Complete GitHub profiles

#11
post #7

Eh, you were too honest in your FAQs - chances are that Github will now close the hole. I expect there is a reason your use-case was not enabled in the API despite being already supported by the official frontend...

ghuser.io contributor here.

We talked with GitHub already. They think that what we are doing is cool and we agreed to tell them our roadmap and they tell us theirs.

We would be glad if GitHub copies us.

Re: Show HN: Ghuser.io – Complete GitHub profiles

#13
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

Isn't 200 requests in an hour ... just three requests per minute? What is this service doing?

Re: Show HN: Ghuser.io – Complete GitHub profiles

#14
post #12

Am I understanding correctly that this is a manual process and not currently automated? (And can I cancel my request so you can move on to others for now?)

It's cheeply automated to handle 10 profile requests per day, which is more than what we got in the past few months. So now we're giving it some human help and we'll have to rethink the system.

What is your username? I'll cancel your request, thanks!

Re: Show HN: Ghuser.io – Complete GitHub profiles

#15
post #4

Earlier quoted context omitted.

This is already more popular than I thought, the queue of profile requests (and AWS SQS) contains already more than 10 users, at this point any new user user coming will have the sensation that nothing happens when they want to get a profile. I didn't expect to reach this point so fast (thanks!). I'll work on improving this right after this wave of new users. For now, just making sure that you all get your profile.

Just requested mine! I'm not ready to abandon GitHub just yet- I use it for the "Hub", not the "Git" and this sort of change is just the kind of thing I'd like to see more of- let's hope some of these ideas get integrated into GitHub itself. I'm still optimistic that Microsoft will be keen to make some quality-of-life improvements when they take the wheel, as a show of good faith.

We had a chat with the "profile team" at GitHub recently and they are doing a great job as we speak at improving the GitHub profiles :)

Re: Show HN: Ghuser.io – Complete GitHub profiles

#16
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

Isn't 200 requests in an hour ... just three requests per minute? What is this service doing?

Maybe it's cloning all GitHub repos of a user to compute activity?

Re: Show HN: Ghuser.io – Complete GitHub profiles

#17
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

Isn't 200 requests in an hour ... just three requests per minute? What is this service doing?

GitHub's API does't provide the exhaustive list of all your contributions. Instead we crawl GitHub's website which is slow.

Details at https://github.com/AurelienLourot/github-contribs#how-does-i...

We are in talks with GitHub and they know that we are crawling GitHub.

Re: Show HN: Ghuser.io – Complete GitHub profiles

#18
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

Isn't 200 requests in an hour ... just three requests per minute? What is this service doing?

this is the bottleneck: https://github.com/AurelienLourot/github-contribs

It uses a very slow non-official GitHub API and so it takes several hours to do the initial crawling of one single profile, and is limited by IP so you need several machines/instances in order to parallelize. We plan to use AWS Fargate for the future. (We thought this future would be much farther away)

Re: Show HN: Ghuser.io – Complete GitHub profiles

#19
post #12

Am I understanding correctly that this is a manual process and not currently automated? (And can I cancel my request so you can move on to others for now?)

Code-wise it is fully automated. But it's slow and it doesn't scale. We have to spin up new servers manually. As OP said, we'll need to make changes for ghuser.io to be scalable. Ideally GitHub adds an API that lists all your contribs. Which is (obviously) not in our hands but we are talking with GitHub.

Re: Show HN: Ghuser.io – Complete GitHub profiles

#20
post #7

Eh, you were too honest in your FAQs - chances are that Github will now close the hole. I expect there is a reason your use-case was not enabled in the API despite being already supported by the official frontend...

ghuser.io contributor here. We talked with GitHub already. They think that what we are doing is cool and we agreed to tell them our roadmap and they tell us theirs. We would be glad if GitHub copies us.

GitHub copying you! Now that's cool! achievement unlocked
Post reply on HN