Live data from Hacker News

Stack Overflow Outage Postmortem

stackstatus.net

151–160 of 335 posts

Re: Stack Overflow Outage Postmortem

#151

They implemented trim with a regex? Neither Java nor .NET do that. The postmortem here should probably be "why are you reimplementing trim".

I like the trim() functions that accept a list (of some form) for 'characters to remove'. It is far more deterministic (better worse case outcome) to follow this approach than it is to run a regexp.

Re: Stack Overflow Outage Postmortem

#152

"This regular expression has been replaced with a substring function." This should be the title of a book on software engineering.

I've fixed so many bugs using regex, only to have to fix several bugs later.

My current stance is, avoid regex if at all possible. Turns out, many of the things we use regex for is possible without. Often times, .Substring, .IndexOf, and using LINQ over strings is sufficient.

Re: Stack Overflow Outage Postmortem

#155
post #67

Earlier quoted context omitted.

This is exactly what we did to diagnose (source: I was on the call). The only tricky part was figuring out which post it was, since it wasn't in the stacktrace. To do that, we grabbed the 3000 most recent posts and ran the regex against them. By that point we already had the code fix (another dev working on it in parallel), but if we hadn't we also could have gotten back up by just deleting the post.

Any idea if it was a malicious attempt? It kind of sounds like it was.

Nah, https://en.wikipedia.org/wiki/Hanlons_razor

If it was malicious they would have made a bunch of them, not just one. I personally have seen many files with unreal amounts of whitespace at the end.

Re: Stack Overflow Outage Postmortem

#156

This is why I always do: s/^\s+//; s/\s+$//; Instead of: s/^\s+|\s+$//; Weirdly, I've "known" this since I started writing Perl in the mid-'90. Not sure where I originally read it (or was told it). Funny how that works. I try to write my regexes such that they anchor at the front of the strong or the back, or they describe the whole string; never an either-or anchoring type situation like this example. Spaces at begi…

You can see in my other post that this doesn't always work. For example, Python's regex engine chokes on `\s+$` if you use `re.search`, but works fine if you use `re.match`.

It's likely that Perl has an optimization for `\s+$` but not `^\s+|\s+$` (the former regex only ever matches at the end of the string, which is amenable to optimizations).

Re: Stack Overflow Outage Postmortem

#157
post #28

Earlier quoted context omitted.

I think you've got the right approach - a vertical slice through the app that checks every layer. You want to know if a user can get useful info from your site, and it tracks (separately!) the common path their query would follow. The danger is that the endpoint becomes public knowledge and comes under a DDOS attack. Putting an IP address filter on that endpoint is usually enough to stop that.

The concept I try to go for with that status check is 'Can this node connect to everything so it can successfully respond to http requests'. However my approach wouldn't identify an overloaded server, which might be a good thing if we need to scale up - taking down an overloaded server is just going to make the other servers that much more overloaded. I'm aways up for hearing about other ways people solve health chec…

Just because it can connect to everything doesn't mean it can successfully respond though.

Re: Stack Overflow Outage Postmortem

#158
post #153

I'm surprised that a developer was able to fix StackOverflow without being able to look up the error message on StackOverflow.

Well, StackOverflow devs are able to cheat and load up the site on their local machine if they want to

I'm not sure I like the idea of being able to connect to a prod db from a dev instance, but whatever floats your boat.

Re: Stack Overflow Outage Postmortem

#159
post #70

> If the string to be matched against contains 20,000 space characters in a row, but not at the end, then the Regex engine will start at the first space, check that it belongs to the \s character class, move to the second space, make the same check, etc. After the 20,000th space, there is a different character, but the Regex engine expected a space or the end of the string. Realizing it cannot match like this it back…

You are wrong and shouldn't speak authoritatively about things you know little about. Most regex engines, including the one in Perl, do backtrack as described. As a special case, Perl will match backwards from a $ (or in look-behind), in specific cases, but not in the case described, where the $ only appears in one part of an alternation. Similarly, as the other poster noted, most regular expression engines default to 'search' as the primitive, rather than matching only from the start.

So, yes, in a sequence of 20,000 characters not appearing at the end of the string, the typical regex engine will backtrack in a manner exposing O(n^2) performance in this case.

If you're going to test something, use the actual regexes they talk about, and perhaps the language/runtime they talk about.

Re: Stack Overflow Outage Postmortem

#160

This is why I always do: s/^\s+//; s/\s+$//; Instead of: s/^\s+|\s+$//; Weirdly, I've "known" this since I started writing Perl in the mid-'90. Not sure where I originally read it (or was told it). Funny how that works. I try to write my regexes such that they anchor at the front of the strong or the back, or they describe the whole string; never an either-or anchoring type situation like this example. Spaces at begi…

You can see in my other post that this doesn't always work. For example, Python's regex engine chokes on `\s+$` if you use `re.search`, but works fine if you use `re.match`. It's likely that Perl has an optimization for `\s+$` but not `^\s+|\s+$` (the former regex only ever matches at the end of the string, which is amenable to optimizations).

I did see your other post, and upvoted it. This rule of thumb has served me well between different regex dialects and implementations, but it's not surprising that there are some specific cases that are "broken" for lack of a better word.

I haven't done much Python but the documentation for re.search() and re.match() is very clear: use search to find an expression anywhere in a string, use match to find an expression at the beginning of a string. It appears to ignore anchors in both cases? Left undetermined then is how to anchor an expression at the end of a string. You say re.match() works, but this is pretty confusingly described, and it's easy to see how the ambiguity can lead to problems for even the most experienced programmers.

Post reply on HN