Live data from Hacker News

Show HN: Ghuser.io – Complete GitHub profiles

github.com

31–40 of 52 posts

Re: Show HN: Ghuser.io – Complete GitHub profiles

#31
post #2

OP here. The idea came because on your GitHub profile: - I can't see all your code contributions since the beginning of time at a glance. - For each of your pinned repos, I can't tell at the first glance whether you just fixed a typo or you wrote 90% of it. ghuser.io tries to close this gap. You can show off with your entire GitHub "portfolio" and it gets refreshed every day. And hopefully this is temporary. The best…

Thanks, my gh profile has become utterly useless due to overload.

Re: Show HN: Ghuser.io – Complete GitHub profiles

#32
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

You may want to delete my profile request, I have somewhere between 50k and 60k repositories and it tends to crash services. (user: sethwoodworth)

Re: Show HN: Ghuser.io – Complete GitHub profiles

#33
post #32
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

You may want to delete my profile request, I have somewhere between 50k and 60k repositories and it tends to crash services. (user: sethwoodworth)

Haha! That is amazing! How, exactly, do you have so many?!

Re: Show HN: Ghuser.io – Complete GitHub profiles

#34
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

Can we clone the repo and run it locally on my own profile? I arrived late..

Re: Show HN: Ghuser.io – Complete GitHub profiles

#35
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

Can we clone the repo and run it locally on my own profile? I arrived late..

Not currently but we are thinking about it, see https://github.com/AurelienLourot/ghuser.io/issues/103

Re: Show HN: Ghuser.io – Complete GitHub profiles

#36

Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well? It would also be really cool if we could pull out basic private repo info as well (with the user's permission). E.g. Worked on X private repos covering 60% Go, 20% Javascript etc. I am also a bit confused as to why you would build this if you don't work at github?

> Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well?" Yes totally. Although we currently think that the best for the community is GitHub improving their GitHub profiles making ghuser.io obsolete. Right now we'd rather do this with GitHub and not against GitHub. But we don't know how much GitHub is willing to help us. (We need changes to the GitHub APIs for ghuser.io to b…

I love these profiles. At GitLab we're very open to contributions to enhance user profiles. One of the hard things might be the performance of these pages, for that you probably need to compute metrics in advance in a background job.

Re: Show HN: Ghuser.io – Complete GitHub profiles

#37

Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well? It would also be really cool if we could pull out basic private repo info as well (with the user's permission). E.g. Worked on X private repos covering 60% Go, 20% Javascript etc. I am also a bit confused as to why you would build this if you don't work at github?

> Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well?" Yes totally. Although we currently think that the best for the community is GitHub improving their GitHub profiles making ghuser.io obsolete. Right now we'd rather do this with GitHub and not against GitHub. But we don't know how much GitHub is willing to help us. (We need changes to the GitHub APIs for ghuser.io to b…

Just so you know, you have a name collision with a popular ClojureScript SPA library, re-frame: https://github.com/Day8/re-frame

Re: Show HN: Ghuser.io – Complete GitHub profiles

#39
post #9

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

Would it be practical to use puppeteer / headless chrome from within a Google Cloud Function to do this? Then you could do literally millions of profiles per hour. The only requirement is that it never takes more than 540 seconds for any one profile, and that you could work around if you write in the ability to export the execution state and send it off to a new function invocation if you are close to the deadline. This seems like a much easier problem if you can get out of VPS land and into lambda/cloud functions since its something like $0.00001 per invocation.

https://cloud.google.com/blog/products/gcp/introducing-headl...

Re: Show HN: Ghuser.io – Complete GitHub profiles

#40
post #14
post #12

Am I understanding correctly that this is a manual process and not currently automated? (And can I cancel my request so you can move on to others for now?)

It's cheeply automated to handle 10 profile requests per day, which is more than what we got in the past few months. So now we're giving it some human help and we'll have to rethink the system. What is your username? I'll cancel your request, thanks!

Seems like the perfect match for aws lambdas. I’d consider setting the tasks for crawling in to SQS and then trigger the lambdas to go do single crawl functions.

For better control over the throttle and concurrency you can leverage dynamodb... I love it for controlling lambdas but not for storage.

If you need more power than a lambda then you can do a similar process with EC2. Populate the SQS Trigger the aws lambda to turn on EC2 machine. Consider spot instances to save a ton of money.

If you need ideas I’m sure HN readers would be glad to help solution for afar.

Post reply on HN