Live data from Hacker News

Stack Overflow Outage Postmortem

stackstatus.net

181–190 of 335 posts

Re: Stack Overflow Outage Postmortem

#181
post #64

They have limits on everything (comments per second, edits per second, upvotes per day, reputation earned per day, etc), it seems like they should have an upper bound character limit on what they accept too.

They do. It's 30K characters. Occasionally you see genuine posts bumping up against it.

Re: Stack Overflow Outage Postmortem

#182
As perlfaq4[1] shows:

    > You can do that with a pair of substitutions:

    >         s/^\s+//;
    >         s/\s+$//;
It then notes, in an understated manner:

    > You can also write that as a single substitution,
    > although it turns out the combined statement is 
    > slower than the separate ones. That might not 
    > matter to you, though:

    >        s/^\s+|\s+$//g;
[1]: http://perldoc.perl.org/perlfaq4.html#How-do-I-strip-blank-s...

Re: Stack Overflow Outage Postmortem

#183
post #153

I'm surprised that a developer was able to fix StackOverflow without being able to look up the error message on StackOverflow.

Well, StackOverflow devs are able to cheat and load up the site on their local machine if they want to

Anyone can download the underlying data and look at it on their local machine.

https://archive.org/details/stackexchange

Re: Stack Overflow Outage Postmortem

#184
post #6

Perfect. Awesome bug. Awesome Post Mortem. This was just fun to read. While this might have been caused by mistake - these types of bugs can be (and are) abused by hackers. https://www.owasp.org/index.php/Regular_expression_Denial_of... https://en.wikipedia.org/wiki/ReDoS The post also links to this video: https://vimeo.com/112065252

Have to agree. I read a Post Incident Review like this and I'm like "Yep, totally see how that could happen. Thanks for solving it and thanks for letting me know why it happened".

Re: Stack Overflow Outage Postmortem

#185

It seems like there should be a way to determine whether a regex can be compiled using the classic O(n) DFA algorithm or with whatever madness PCREs use to support backtracking and so on. Anybody know if any regex engines attempt this? Obviously you can still shoot yourself in the foot, but it's somewhat more difficult to do so in a situation like this where the regex in question "looks" cheap.

> It seems like there should be a way to determine whether a regex can be compiled using the classic O(n) DFA algorithm or with whatever madness PCREs use to support backtracking and so on. Obviously. If the "regex" includes a backreference, it requires backtracking. If it includes only regular operations (I can't call any other nonregular operations that people might expect to mind), it doesn't. This is information…

I wonder why regex libraries don't do this then? Isn't /\s+$/ backreference free? Or does including the anchor change that?

Re: Stack Overflow Outage Postmortem

#186
post #158
post #153

Earlier quoted context omitted.

Well, StackOverflow devs are able to cheat and load up the site on their local machine if they want to

I'm not sure I like the idea of being able to connect to a prod db from a dev instance, but whatever floats your boat.

There are public dumps of Stack Overflow's database. https://archive.org/details/stackexchange

Re: Stack Overflow Outage Postmortem

#188

Earlier quoted context omitted.

I did see your other post, and upvoted it. This rule of thumb has served me well between different regex dialects and implementations, but it's not surprising that there are some specific cases that are "broken" for lack of a better word. I haven't done much Python but the documentation for re.search() and re.match() is very clear: use search to find an expression anywhere in a string, use match to find an expression…

Anchors aren't ignored. For re.match, `^abc$` is equivalent to `abc`, so the anchors are just redundant. (N.B. `^^abc$$` will match the same set of strings as `^abc$`.) For re.search, `^abc$` is equivalent to `re.match(^abc$)` but `abc` is equivalent to `re.match(.STAR?abc.STAR?)`. But yes, this is a very subtle semantic and different regex engines handle it differently. My favorite approach is to always use `.STAR?t…

Correction: re.match anchors at the start, but not the end, so `^abc$` is equivalent to `abc$` but not `abc`.

Re: Stack Overflow Outage Postmortem

#189

> It took 10 minutes to identify the cause. I'm impressed they were able to do this so quickly.

Agreed that's impressive debugging for this issue. But...

> 10 minutes to roll out the fix

That seems very slow to me. 30% of their down time was because their deploy process is slow.

Re: Stack Overflow Outage Postmortem

#190
post #165
post #45

Earlier quoted context omitted.

Yeah, I'm trying to figure out how you even get 20,000 spaces into a Stack Exchange post, and how it would render in your browser.

It was in a multiline code block, so it just had a tonne of horizontal scroll. See the edit: http://stackoverflow.com/revisions/38484433/2

Good find! Where did you see this? Or was it in the comments via Nick Craver?
Post reply on HN