Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stack Overflow Outage Postmortem

stackstatus.net

Post reply on HN
Print view View original

241–250 of 335 posts

Re: Stack Overflow Outage Postmortem

#241

Post by danek »

Someone1234 wrote: post #36
dkopi wrote: post #6

Perfect. Awesome bug. Awesome Post Mortem. This was just fun to read. While this might have been caused by mistake - these types of bugs can be (and are) abused by hackers. https://www.owasp.org/index.php/Regular_expression_Denial_of... https://en.wikipedia.org/wiki/ReDoS The post also links to this video: https://vimeo.com/112065252

Well in this case a post contained 20K whitespaces, so I wouldn't jump to the conclusion that it was a mistake rather than intentional.

I wouldn't jump to any conclusion ;)

But I do know that tools often make it easy for people do incredibly stupid things by accident. Like the 'sudo remove file' command followed by '-rf' in the wrong place. I rimraf a lot; it's very useful. It's a miracle I haven't wiped out any computers doing so...

Re: Stack Overflow Outage Postmortem

#243

Post by htfy96 »

jldugger wrote: post #177
krabpaaltje wrote: post #138

Earlier quoted context omitted.

Bottle of tequila on the spacebar

Eh, just type a single space. Then copy the entire post and paste it a couple dozen times.

Only 35 keystrokes are required to get 20K+ spaces.[1]

[1] https://stackoverflow.com/questions/4606984/maximum-number-o...

Re: Stack Overflow Outage Postmortem

#244

Post by ww520 »

gibrown wrote: post #189
kilroy123 wrote: post #23

> It took 10 minutes to identify the cause. I'm impressed they were able to do this so quickly.

Agreed that's impressive debugging for this issue. But... > 10 minutes to roll out the fix That seems very slow to me. 30% of their down time was because their deploy process is slow.

10 minutes roll out to production is insanely fast. Roll out usually goes through build, test, staging, and to farms of production servers, with smoke tests in each stage along the way.

Re: Stack Overflow Outage Postmortem

#245

Post by gnuvince »

mattmanser wrote: post #192
judah wrote: post #152

Earlier quoted context omitted.

I've fixed so many bugs using regex, only to have to fix several bugs later. My current stance is, avoid regex if at all possible. Turns out, many of the things we use regex for is possible without. Often times, .Substring, .IndexOf, and using LINQ over strings is sufficient.

Regexes are almost always a massive code smell. They should almost never be used, bad idea, bad implementation, hard to grok, hard to debug, hard to test, hard to spot. Whoever came up with them has surely been given the same honorary place in hell with Jon Postel, who invented the utterly disastrous "be liberal in what you accept", that has plagued all web developers for the last 20 years.

> Regexes are almost always a massive code smell. They should almost never be used

Regular expressions are just a notation for expressing a finite state automata that recognizes a regular language: if that's the problem that you have, regular expressions are definitely the tool you want to be using. For instance, I recently made myself a small tool to compute the min/max/expected value of a dice roll; the notation for such a thing forms a regular language that can be expressed as follows:

    non_zero_digit ::= '1' | ... | '9'
    digit ::= '0' | non_zero_digit
    integer ::= non_zero_digit digit*
    modifier ::= '+' integer | '-' integer
    roll ::= integer 'd' integer modifier?
Converting this grammar to a regular expression is straight-forward and was the correct tool to use.

I agree that regular expressions are often used in contexts where they should not, especially when people start using back-references to recognize non-regular languages, but don't throw out a perfectly good tool because sometimes it is ill-suited.

Re: Stack Overflow Outage Postmortem

#247

Post by superuser2 »

xigency wrote: post #78
chubot wrote: post #9

Ha! The same bug happened internally at my company. In that case it was a regex matching a URL taking so much CPU as to cause a DOS of a proxy server. I won't be surprised if it's happened to someone here too. This is very timely, because minutes ago, I made a link to Russ Cox's articles in my Kernighan awk repo: https://github.com/andychu/bwk https://swtch.com/~rsc/regexp/regexp1.html If you are not familiar with th…

The key quote here is: "Regular expressions are one of computer science's shining examples of how using good theory leads to good programs ..." "Today, regular expressions have also become a shining example of how ignoring good theory leads to bad programs. The regular expression implementations used by today's popular tools are significantly slower than the ones used in many of those thirty-year-old Unix tools." The…

The name regular expression seems meant to evoke regular languages, whose significance is that they can be recognized in a very straightforward way (by finite state automata) without pathological gotcha cases like this.

Perhaps we ought to call them irregular expressions - this kind of behavior is the literal antithesis of what regular means in automata/parsing theory.

Re: Stack Overflow Outage Postmortem

#248

Post by wpears »

flukus wrote: post #202
mattmanser wrote: post #192

Earlier quoted context omitted.

Regexes are almost always a massive code smell. They should almost never be used, bad idea, bad implementation, hard to grok, hard to debug, hard to test, hard to spot. Whoever came up with them has surely been given the same honorary place in hell with Jon Postel, who invented the utterly disastrous "be liberal in what you accept", that has plagued all web developers for the last 20 years.

How are they hard to test? Given input x, expect output y. It's one of the easiest things in the world to test.

Where x can be any combination of letters of any length?

Re: Stack Overflow Outage Postmortem

#249

Post by scrollaway »

hinkley wrote: post #217
OJFord wrote: post #165

Earlier quoted context omitted.

It was in a multiline code block, so it just had a tonne of horizontal scroll. See the edit: http://stackoverflow.com/revisions/38484433/2

If he had used tabs instead of spaces this wouldn't have been a problem.

Or at the very least, trim whitespace on save :)

Re: Stack Overflow Outage Postmortem

#250

Post by solipsism »

dingo_bat wrote: post #234
solipsism wrote: post #220

Earlier quoted context omitted.

My guess is they're searching for the first non-whitespace character, reverse-searching for the last non-whitespace character, and then using String.Substring to return only what's in the middle. As to why they're not using String.Trim ( https://msdn.microsoft.com/en-us/library/t97s7bs3(v=vs.110).... ), maybe it's because String.Trim doesn't seem to know about the 200c whitespace character.

From what I understood, trim would work perfectly. It's 200 spaces not a single 200 width character.

You're misreading the regex. \u200c is a single whitespace character. http://www.fileformat.info/info/unicode/char/200c/index.htm
Post reply on HN