Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updated rate limits for unauthenticated requests

github.blog

Post reply on HN
Print view View original

81–90 of 187 posts

Re: Updated rate limits for unauthenticated requests

#81

Post by zarzavat »

blinker21 wrote: post #51
gnabgib wrote: post #3

60 req/hour for unauthenticated users 5000 req/hour for authenticated - personal 15000 req/hour for authenticated - enterprise org According to https://docs.github.com/en/rest/using-the-rest-api/rate-limi... I bump into this just browsing a repo's code (unauth).. seems like it's one of the side effects of the AI rush.

I've hit this over the past week browsing the web UI. For some reason, github sessions are set really short and you don't realise you're not logged in until you get the error message. I really wish github would stop logging me out.

Yes, it's not the rate limits that are the problem per se but GitHub's tendency to log you out and make you go through 2fa.

If they would let me stay logged in for a year then I wouldn't care so much.

Re: Updated rate limits for unauthenticated requests

#82

Post by dghlsakjg »

blinker21 wrote: post #51
gnabgib wrote: post #3

60 req/hour for unauthenticated users 5000 req/hour for authenticated - personal 15000 req/hour for authenticated - enterprise org According to https://docs.github.com/en/rest/using-the-rest-api/rate-limi... I bump into this just browsing a repo's code (unauth).. seems like it's one of the side effects of the AI rush.

I've hit this over the past week browsing the web UI. For some reason, github sessions are set really short and you don't realise you're not logged in until you get the error message. I really wish github would stop logging me out.

Something strange is going on. I think GH has kept me logged in for months at a time. I honestly can’t remember the last time I had to authenticate.

Re: Updated rate limits for unauthenticated requests

#83

Post by 1oooqooq »

Novosell wrote: post #54
blinker21 wrote: post #51

Earlier quoted context omitted.

I've hit this over the past week browsing the web UI. For some reason, github sessions are set really short and you don't realise you're not logged in until you get the error message. I really wish github would stop logging me out.

Hmmmm, Github keeps me logged in for months I feel like. Unless I'm misunderstanding the github security logs, my current login is since march.

GH is Microsoft's most successful social network.

GH now uses the publisher business model, and as such, they lose money when you're logged out. same reason why google, fb, etc will not ask you for a password for decades.

Re: Updated rate limits for unauthenticated requests

#84

Post by hardwaresofton »

Does it seem to anyone like eventually the entire internet will be login only?

At this point knowledge seems to be gathered and replicated to great effect and sites that either want to monetize their content OR prevent bot traffic wasting resources seem to have one easy option.

Re: Updated rate limits for unauthenticated requests

#85

Post by zarzavat »

notatoad wrote: post #77
ikiris wrote: post #72

Earlier quoted context omitted.

1/min? That’s insanely low.

60/hr is not the same as 1/min, unless you're trying to continually make as many requests as possible, like a crawler. and if that is for your use case, then your traffic is probably exactly what they're trying to block.

60/h is obviously well within normal human usage of an app and not bot traffic...

A normal rate limit to separate humans and bots would be something like 60 per minute. So it's about an order of magnitude too low.

Re: Updated rate limits for unauthenticated requests

#86

Post by simoncion »

latentsea wrote: post #76
dmitrygr wrote: post #64

Earlier quoted context omitted.

Criminal charges under CFAA to actual CEOs of actual companies doing this, with long jail terms.

I don't know if I ever recall seeing a CEO go to jail for practically anything, ever. I'm sure there are lots of examples, but at this point in my life I have kind of derived a rule of thumb of "if you want to commit a crime, just disguise it as a legitimate business" based off seeing so many times where CEOs get off scott free .

FWIW, Joseph Nacchio (the CEO of Qwest Communications) went to jail for like a decade for refusing to help the NSA violate FISA circa 2001.

Re: Updated rate limits for unauthenticated requests

#87

Post by mjevans »

zarzavat wrote: post #85
notatoad wrote: post #77

Earlier quoted context omitted.

60/hr is not the same as 1/min, unless you're trying to continually make as many requests as possible, like a crawler. and if that is for your use case, then your traffic is probably exactly what they're trying to block.

60/h is obviously well within normal human usage of an app and not bot traffic... A normal rate limit to separate humans and bots would be something like 60 per minute . So it's about an order of magnitude too low.

Use case: crawling possibly related files based on string search hints in a repo you know nothing about...

Something on the order of 6 seconds a page doesn't sound TOO out of human viewing range depending on how quickly things load and how fast rejects are identified.

I could see ~10 pages / min which is 600 pages / hour. I could also see the argument that a human would get tired at that rate and something closer to 200-300 / hr is reasonable.

Re: Updated rate limits for unauthenticated requests

#88

Post by mjevans »

hardwaresofton wrote: post #84

Does it seem to anyone like eventually the entire internet will be login only? At this point knowledge seems to be gathered and replicated to great effect and sites that either want to monetize their content OR prevent bot traffic wasting resources seem to have one easy option.

Static, Near Static (not generated on demand at least; generated only on real content update), and Login seems likely.

AI not caching things is a real issue. Sites being difficult TO cache / failing the 'wget mirror test' is the other side of the issue.

Re: Updated rate limits for unauthenticated requests

#89

Post by koolba »

ikiris wrote: post #73
londons_explore wrote: post #62

Earlier quoted context omitted.

Surely most AI trawlers have special support for git and just clone the repo once?

I think you vastly overestimate the average dev and their care for handling special cases that are mostly other people’s aggregate problem.

Can’t they use the AIs to do it?

Re: Updated rate limits for unauthenticated requests

#90

Post by grishka »

mjevans wrote: post #88
hardwaresofton wrote: post #84

Does it seem to anyone like eventually the entire internet will be login only? At this point knowledge seems to be gathered and replicated to great effect and sites that either want to monetize their content OR prevent bot traffic wasting resources seem to have one easy option.

Static, Near Static (not generated on demand at least; generated only on real content update), and Login seems likely. AI not caching things is a real issue. Sites being difficult TO cache / failing the 'wget mirror test' is the other side of the issue.

What about AI not respecting robots.txt? I myself have never ran into this, but I've seen complaints of many people who did.
Post reply on HN