Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stack Overflow Outage Postmortem

stackstatus.net

Post reply on HN
Print view View original

81–90 of 335 posts

Re: Stack Overflow Outage Postmortem

#81

Post by KyleBrandt »

jandrese wrote: post #56
zymhan wrote: post #45

Earlier quoted context omitted.

Yeah, I'm trying to figure out how you even get 20,000 spaces into a Stack Exchange post, and how it would render in your browser.

Browsers typically collapse whitespace, so it probably would render as a single space.

was in code tags, so had a very long horizontal scroll bar :P

Re: Stack Overflow Outage Postmortem

#82

Post by maxgee »

revelation wrote: post #52
gortok wrote: post #47

Earlier quoted context omitted.

You can't trust that the user has JavaScript enabled; and since you can't trust user input anyway, you'd have to do this server side.

"You can't trust user input" means to not fudge with user data in the first place! If your code cares about unicode spaces at the beginning of user input, there is your problem. (Also, you missed the reject part)

How would you suggest running a user-created-content site without ever fudging/sanitizing the data being posted to the server?

Re: Stack Overflow Outage Postmortem

#83

Post by mfukar »

StavrosK wrote: post #29

I don't understand something: the regex expected a space character, followed by the end of the string. If the last character wasn't a space, this could never match. Why did the engine keep backtracking, even though it's easy to figure out that it could never match the regex?

Because if the engine only examined each character at most once, then it would not be a backtracking RE engine.

Re: Stack Overflow Outage Postmortem

#84

Post by oddx »

kilroy123 wrote: post #23

> It took 10 minutes to identify the cause. I'm impressed they were able to do this so quickly.

I guess it was just:

- login to server

- make dump of all threads stack traces

- see that something like Regexp.match present in all stacks

- find function that called this regexp.

Re: Stack Overflow Outage Postmortem

#86

Post by KyleBrandt »

revelation wrote: post #30

They implemented trim with a regex? Neither Java nor .NET do that. The postmortem here should probably be "why are you reimplementing trim".

Very old code, hard to say what the thinking was at the time. That is why we are doing an audit looking for any unneeded regexes, or regexes that are susceptible to backtracking

Re: Stack Overflow Outage Postmortem

#89

Post by smegel »

StavrosK wrote: post #29

I don't understand something: the regex expected a space character, followed by the end of the string. If the last character wasn't a space, this could never match. Why did the engine keep backtracking, even though it's easy to figure out that it could never match the regex?

You make a valid point. Either their analysis is dodgy, or the regex engine is very, very poor.

Re: Stack Overflow Outage Postmortem

#90

Post by deadowl »

alexbecker wrote: post #77

I remember the day I learned that Python's "re" module uses backtracking for non-extended regexes. My tests covered lots of corner cases in the regex logic, but were too short for me to notice the performance penalty. Luckily I only caused a partial outage in production. I actually got to talk to Raymond Hettinger (Python core team) about why re uses a potentially exponential-time algorithm for regexes when there is…

I looked at that module a while back. The goto statements make it very difficult to analyze. It seems to integrate well with the signals module, which could be used to interrupt a large class of ReDoS attempts, but determining whether all potential regexps could be interrupted using signals, with the goto statements, is a difficult task.
Post reply on HN