41–50 of 52 posts
Given you disabled onboarding, is my onboarding request queued somewhere or will I have to comeback and re-request once you got over the hump?
Given you disabled onboarding, is my onboarding request queued somewhere or will I have to comeback and re-request once you got over the hump?
We're working on serving all the requests that came in before we disabled onboarding (about 200 requests). If you give me your github username I can tell you if you're part of them :)
If you have user requests in the logs, you should queue them up.
Pretty bummed I missed a chance to try this! I had manually gone through my merged PRs list before to try and assemble a count of my contributions to each public project to come up with some sort of profile of stuff I've participated in/contributed to, as I really didn't see any of these type of pages doing that. This looks much more like what I'd like to see.
OP here. The idea came because on your GitHub profile: - I can't see all your code contributions since the beginning of time at a glance. - For each of your pinned repos, I can't tell at the first glance whether you just fixed a typo or you wrote 90% of it. ghuser.io tries to close this gap. You can show off with your entire GitHub "portfolio" and it gets refreshed every day. And hopefully this is temporary. The best…
I'm the guy fixing the typos in everybody's README :)
OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!
Would it be practical to use puppeteer / headless chrome from within a Google Cloud Function to do this? Then you could do literally millions of profiles per hour. The only requirement is that it never takes more than 540 seconds for any one profile, and that you could work around if you write in the ability to export the execution state and send it off to a new function invocation if you are close to the deadline. T…
Earlier quoted context omitted.
> Instead we crawl GitHub's website which is computationally expensive I'm surprised to hear that you're bottlenecking on CPU time. Could you verify that my understanding is correct? I would've thought your bottleneck would be networking and connectivity as you have to wait for GitHub to process all of the requests.
You're right. I edited my answer. The GitHub's unofficial API we are using is slow and per IP rate limited. We spin up several servers to have several IPs to circumvent the rate limit. (GitHub knows that we do that and we are in contact with them.)
Surely a rate-limit exception could be in order, then?
And perhaps you could help them alpha-test a new API endpoint that just so happens to include all the info rolled up as one URL :D
(Hmmmmm.... GraphQL....)
Earlier quoted context omitted.
Would it be practical to use puppeteer / headless chrome from within a Google Cloud Function to do this? Then you could do literally millions of profiles per hour. The only requirement is that it never takes more than 540 seconds for any one profile, and that you could work around if you write in the ability to export the execution state and send it off to a new function invocation if you are close to the deadline. T…
That's amazingly cheap, but would each puppeteer instance have its own IP address? They're being rate limited on each IP.
Check out https://sourcerer.io - GitHub and GitLab profiles with extensive timeline, language, library, repo analysis. Customizable.