261–270 of 804 posts
Earlier quoted context omitted.
Strange to think they are probably triaged by LLMs at this point
AI finding issues in code and reporting them so that an AI can review and triage them for another AI to fix.
I don't know if that's sarcasm or not. I know it doesn't work, but that's the future we've been promised, right?
Are retries bad? These are the sort of reason they make me generally uncomfortable. I appreciate they might be useful in scenarios where connectivity is inherently problematic (e.g. mobile connectivity), but for a super connected and very desktoppy service I'd rather not retry much, if at all. As it obscures it when stuff has genuinely gone wrong, and this worst case scenario is tragic. I feel like I'm mildly stupid…
I totally agree with you, I think retries are overused, with the exception of operations that are known to be unreliable and can't be improved. In my experience, errors which go away within a few seconds are quite rare, and are mainly due to flaws which are usually caught in testing. I think a very careful cost/risk/benefit analysis should be done when adding automatic retries to things. As well as potentially causin…
In particular, networks tend to be a LOT less reliable than the typical developer accounts for. And the failures are very often transient. A case I run into often is doing something with my phone while leaving the house. There’s a window where it still thinks it’s on the WiFi but it’s too far away for it to work anymore. Initiating an action in that window often produces an alert telling me to try again, and trying again a few seconds later almost always works.
Earlier quoted context omitted.
I'm not following your line of questioning. Without ever using github as an online editing platform, you can do one push with two new commits.
Commits are not expensive, pushes are. You can do any number of commits before you do one push, unless you are editing online, in which case every act is it's own commit & push. You can rig up a local ide to pathologically commit+push per save, but you can do literally anything, so what you can do is immaterial.
I think it's because their leadership actually doesn't care that it goes down. A weekly outage is now an accepted cost of continuing to allow unlimited free access with infrastructure that cannot possibly handle the load. As a result, everyone is looking at their GitHub Enterprise bills and cost of stopped work, calculating how much they'd save by self-hosting.
I still can't help but feel a little grateful for what they do across the free side of things. I know it isn't altruism, and I know nobody needs to defend a billion dollar corporation but...
Name another service that does what they do for FREE (and no ads) at this scale. It isn't easy. Wikipedia has probably more usage, but is a simpler endeavor. (except the moderation part, that's just amazing) Open Street map? Smaller and simpler. Internet archive? Again, smaller and simpler. Linux distro mirrors? Again, smaller and simpler than whatever github is doing for free.
Everyone suggesting that they simply charge users for commits to drive off AI-heavy users forgets that Github is owned by Microsoft, who has a big incentive to keep having developers use AI. I suspect that Microsoft would even prefer to have Github operate at a loss, if that loss were because all its users were using their models and paying for OpenAI subscriptions to generate the code.
> I suspect that Microsoft would even prefer to have Github operate at a loss, I assumed it does , do you know that it doesn't?
Earlier quoted context omitted.
Exponential backoff is the wrong answer in a highly available system in the typical case where (a) failure is expected and (b) you have nodes you are supposed to fail over to.
Your highly available system is probably somewhat important, otherwise you won’t have invested in making it HA. While your premise holds for happy cases, when you do have a cascading series of outages, not using exponential backoff is just adding a self-inflicted DoS to when you do go down. I don’t really follow your premise and can’t really articulate many cases for when you shouldn’t use exponential backoff. Maybe…
To handle this correctly you need your RPC framework to accurately communicate retryable vs non-retryable failures to clients. Then service A knows service B is dead, does not retry, and proapgates the failure to clients. This is hard to do perfectly, but there's no alternative that works.
Earlier quoted context omitted.
Jittered exponential backoff. You don't want the whole herd to come back at the same time, you have to add timing jitter to the clients.
Possibly dumb/silly question.... Are there any sorts of reverse proxies out there that provide their own layer of jittered/exponential backoff based on patterns? (i.e. requesting IP, cookie, etc.) I suppose the main reason I think it might be a bad idea, is that it would add complexity to the reverse proxy (i.e. now it's having to track whatever thing is being used and that complexity itself becomes a potential failu…
Earlier quoted context omitted.
The right answer is for the RPC framework to accurately communicate "try again on another node" vs "don't try again, just hard fail". When one end user request fans out to hundreds of backend requests (typical for microservices), you can't have each of those backend requests do its own exponential backoff. If they do it in parallel, they're a thundering herd, and if they do it in serial, the end user request will tim…
This is why you have circuit breakers upstream. Not on every individual instance.