Stack Overflow Outage Postmortem

Post by tom-lord » Thu, Jul 21, 2016, 9:05 AM UTC

xigency wrote: post #114

xigency wrote: post #78

Earlier quoted context omitted.

The key quote here is: "Regular expressions are one of computer science's shining examples of how using good theory leads to good programs ..." "Today, regular expressions have also become a shining example of how ignoring good theory leads to bad programs. The regular expression implementations used by today's popular tools are significantly slower than the ones used in many of those thirty-year-old Unix tools." The…

Unfortunately, there is a hint as to how this has happened: "This strategy is no longer practical: users have come to rely on backreferences for at least occasional use, and backreferences are part of the POSIX standard for regular expressions." What better excuse is there for a poor implementation than standards compliance? In many ways, using regex with backtracking by default is like programming in Lisp without ta…

Post by webtechgal » Thu, Jul 21, 2016, 9:14 AM UTC

oasisbob wrote: post #201

redbeard0x0a wrote: post #13

In the past, I have done Load Balancer status checks against a special /status endpoint. I queried all the connected services (i.e. DB, Redis, etc) with a super fast query (i.e. `SELECT version();`). Monitoring CPU/MEM usage for scaling was separate. Comparing this to checking the home page, what is the best way to setup a health check for your load balancers?

I'd be VERY careful about including external dependancies in an HTTP health check which results in a web server being removed from service - it's usually an invitation for cascading failures. 1) If you do have a back-end failure, this setup can cloud the root cause during recovery because your downstream web servers are down as well. 2) Transitory back-end failures can cascade, and your health checks can make this wo…

Post by curryhoward » Thu, Jul 21, 2016, 9:21 AM UTC

dingo_bat wrote: post #232

smrtinsert wrote: post #60

"This regular expression has been replaced with a substring function." This should be the title of a book on software engineering.

Also, removing spaces from the start and end of a line is a fairly standard operation which is provided by every language's built in library.

Post by d0mine » Thu, Jul 21, 2016, 9:48 AM UTC

mark-r wrote: post #148

alexbecker wrote: post #77

I remember the day I learned that Python's "re" module uses backtracking for non-extended regexes. My tests covered lots of corner cases in the regex logic, but were too short for me to notice the performance penalty. Luckily I only caused a partial outage in production. I actually got to talk to Raymond Hettinger (Python core team) about why re uses a potentially exponential-time algorithm for regexes when there is…

I vaguely remember a replacement for Python's re module, but I can't remember the name, and Googling for it is an exercise in frustration. Edit: it's "regex" - https://pypi.python.org/pypi/regex . I have no idea if it behaves better with backtracking, a cursory glance at the documentation doesn't indicate any changes in that area.

  $ python -mtimeit -s "s=' '*20000; import regex as re" "re.search(r'\s+$', s)" # ends with spaces is fast
  10000 loops, best of 3: 201 usec per loop
  $ python -mtimeit -s "s=' '*20000+'non-space'; import regex as re" "re.search(r'\s+$', s)" # ends with a non-space is slow
  10 loops, best of 3: 3.39 sec per loop

Post by lazyjones » Thu, Jul 21, 2016, 10:41 AM UTC

smegel wrote: post #70

> If the string to be matched against contains 20,000 space characters in a row, but not at the end, then the Regex engine will start at the first space, check that it belongs to the \s character class, move to the second space, make the same check, etc. After the 20,000th space, there is a different character, but the Regex engine expected a space or the end of the string. Realizing it cannot match like this it back…

  #!/usr/bin/perl

  use Benchmark;

  sub trim {
      my $s=shift;
      $s =~ s/^[\s\u200c]+|[\s\u200c]+$//g;
      return $s;
  }

  timethis(100, sub { trim("x" . " " x 20000 . "x"); });