Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go Language – Web Application Secure Coding Practices

checkmarx.gitbooks.io

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Go Language – Web Application Secure Coding Practices

#31

Post by buro9 »

MajesticHobo wrote: post #29
lomnakkus wrote: post #26

Earlier quoted context omitted.

> As long as the data is sanitized before it can affect the storage/transport mechanism for its content type, you're good. No, not really. Storing the user's data as is is almost always of paramount importance. The fact that it may be output as HTML/XML/MarkDown/whatever means that it really is at output-time that you must sanitize/escape/quote. That's why the moral of the Bobby Tables story isn't: "Oh, just remove a…

I don't disagree with sanitizing data at output time when it's clear that A) the input won't affect anything else and B) output is going to happen . But realize not all input winds up in a SQL database, not all input will be considered valid in all contexts, and not all input eventually becomes output. Sometimes, data really does need to be sanitized at the point of submission. If you disagree, that's more of a point…

You're all saying the same thing.

I didn't specify whether the sanitize occurred on receiving user input or displaying it.

I only said, sanitize all user input.

Re: Go Language – Web Application Secure Coding Practices

#32

Post by curun1r »

buro9 wrote: post #31
MajesticHobo wrote: post #29

Earlier quoted context omitted.

I don't disagree with sanitizing data at output time when it's clear that A) the input won't affect anything else and B) output is going to happen . But realize not all input winds up in a SQL database, not all input will be considered valid in all contexts, and not all input eventually becomes output. Sometimes, data really does need to be sanitized at the point of submission. If you disagree, that's more of a point…

You're all saying the same thing. I didn't specify whether the sanitize occurred on receiving user input or displaying it. I only said, sanitize all user input.

> I didn't specify whether the sanitize occurred on receiving user input or displaying it.

I'm sorry, but you basically did. You said:

> You must sanitize ALL user input even if you don't think you're going to render it on a web page

Which implies that sanitizing input at display time, when you know you're rendering it to a web page, is too late. That's why people are jumping on you. Keeping a clean database is the absolute most important thing you can do. The database isn't contextual. The data it stores can find its way into HTML pages, REST responses, SQL queries, PDF reports, XML/JSON data exports and a ton of other formats. Each of these output formats will require a different form of sanitizing. Sanitizing before the data hits disk creates a nightmare for anyone displaying the data in a context other than the sanitization that was performed. So what you said originally is precisely incorrect. Only sanitize input when you know it's going to be rendered to a webpage. Otherwise, leave it alone.

Now, you should be using view-layer frameworks to make that sanitization easy, automatic and the default action. When rendering to HTML, the templating language should sanitize by default and give a way for template authors to opt-out when they know the data did not come from user input. Likewise, in the SQL context, prepared statements also make it easy for the developer to do the right thing. But at no point are you speculatively sanitizing all user input. You're getting user input to disk in as pristine a format as possible and sanitizing contextually depending on how the data is outputted.

Re: Go Language – Web Application Secure Coding Practices

#33

Post by lomnakkus »

MajesticHobo wrote: post #29
lomnakkus wrote: post #26

Earlier quoted context omitted.

> As long as the data is sanitized before it can affect the storage/transport mechanism for its content type, you're good. No, not really. Storing the user's data as is is almost always of paramount importance. The fact that it may be output as HTML/XML/MarkDown/whatever means that it really is at output-time that you must sanitize/escape/quote. That's why the moral of the Bobby Tables story isn't: "Oh, just remove a…

I don't disagree with sanitizing data at output time when it's clear that A) the input won't affect anything else and B) output is going to happen . But realize not all input winds up in a SQL database, not all input will be considered valid in all contexts, and not all input eventually becomes output. Sometimes, data really does need to be sanitized at the point of submission. If you disagree, that's more of a point…

> But realize not all input winds up in a SQL database, not all input will be considered valid in all contexts, and not all input eventually becomes output.

That was the point I was trying to make: Sanitizing input is fail-from-the-start. There's no way to know ahead of time what outputs you're going to be producing 5 years from now. Conclusion: Store all input exactly as received. (We can do that these days with form/url encodings and whatnot).

Ok, so now you have the data stored accurately.

Next step: You need to output to, let's say, HTML. Ok, so you just escape/quote everything appropriately and nobody gets hurt. If you just do the escaping/quoting properly there is no XSS attacks. It's really just that simple.

However, it is NOT about sanitizing at the "input" point. Do you get what I'm saying now?

(I realize that that sounds aggressive, but I really just want to force this point home. Please tell me if you disagree or find some detail in my explanation confusing. This is important for the security of the web and either I'm wrong or you're wrong or I didn't understand what you said. Let's figure out which is the case.)

[1] There are caveats here.

Re: Go Language – Web Application Secure Coding Practices

#34

Post by lomnakkus »

MajesticHobo wrote: post #29
lomnakkus wrote: post #26

Earlier quoted context omitted.

> As long as the data is sanitized before it can affect the storage/transport mechanism for its content type, you're good. No, not really. Storing the user's data as is is almost always of paramount importance. The fact that it may be output as HTML/XML/MarkDown/whatever means that it really is at output-time that you must sanitize/escape/quote. That's why the moral of the Bobby Tables story isn't: "Oh, just remove a…

I don't disagree with sanitizing data at output time when it's clear that A) the input won't affect anything else and B) output is going to happen . But realize not all input winds up in a SQL database, not all input will be considered valid in all contexts, and not all input eventually becomes output. Sometimes, data really does need to be sanitized at the point of submission. If you disagree, that's more of a point…

(Just to be clear: This is just for argument's sake... and hopefully for edumacation.)

> But realize not all input winds up in a SQL database, not all input will be considered valid in all contexts, and not all input eventually becomes output.

That was the point I was trying to make: Sanitizing input is fail-from-the-start. There's no way to know ahead of time what outputs you're going to be producing 5 years from now. Conclusion: Store all input exactly as received. (We can do that these days with form/url encodings and whatnot).

Ok, so now you have the data stored accurately.

Next step: You need to output to, let's say, HTML. Ok, so you just escape/quote everything appropriately and nobody gets hurt. If you just do the escaping/quoting properly there is no XSS attacks. It's really just that simple.

However, it is NOT about sanitizing at the "input" point. Do you get what I'm saying now?

(I realize that that sounds agressive, but I really just want to force this point home. Please tell me if you disagree or find some detail in my explanation confusing. This is important for the security of the web and either I'm wrong or you're wrong or I didn't understand what you said. Let's figure out which is the case.)

[1] There are caveats here.

Re: Go Language – Web Application Secure Coding Practices

#35

Post by taeric »

osteele wrote: post #13
buro9 wrote: post #3

When I see anything that touches web coding practices for Go I always look up an area that I know best: sanitization. I wrote https://github.com/microcosm-cc/bluemonday which is a pure Go HTML sanitizer inspired by https://github.com/owasp/java-html-sanitizer . The key things to understand about HTML sanitizers: * They must be whitelist based * They must be aware of context * You must sanitize ALL user input even if…

> You must sanitize ALL user input even if you don't think you're going to render it on a web page. I'm not able to make sense of this. Sanitize it for what context? SQL? JSON? HTML? Inclusion as a command-line argument? All of these, and hope that sanitizing it for one context doesn't un-sanitize it for others?

I've always had issue with this advice for this very reason. Worse, when people sanitize in and out of the database. Double encoded html just cracks me up.

I'm a big fan of not necessarily sanitizing, but treating it appropriate in context. This may mean removing characters, or mapping them, or just delimiting the entire thing.

To that end, I argue that you should not necessarily sanitize on the way to the storage mechanism. You should only sanitize at the boundaries. So, a web view should make sure any strings are treated as strings. A database layer should make sure query parameters are not able to alter the query. Etc.

(All of this is trying to simply reinforce your point.)

Re: Go Language – Web Application Secure Coding Practices

#36

Post by richardknop »

coldtea wrote: post #15
MajesticHobo wrote: post #14

Earlier quoted context omitted.

Yes, really. Otherwise, you must take extra care not to reflect any input data back in any response to the user, whether it's in the HTML body or not. See: HTTP response splitting.

No, not really. In fact, you have it totally backwards: you're not supposed to sanitize all user input before storing it. Instead you're supposed to sanitize any user input before you output it back to your webpage. Even more so: it's the output that dictates what sanitization you should perform, not the input. You don't do input sanitization for HTML (for XSS etc) when you store your data in your DB. Instead you sho…

Sanitization does not refer to escaping HTML entities before rendering to avoid cross site scripting. That is just escaping of entities done when rendering templates.

I believe by sanitization most people mean processing content which will be rendered and not escaped, a good example is content from WYSIWYG editors. And this is where sanitization libraries would come into play.

You would sanitize HTML fragments before storing them in database because you don't escape them during rendering. Text content is not sanitized before saving to database as you can just escape it when rendering.

Re: Go Language – Web Application Secure Coding Practices

#38

Post by pram »

buro9 wrote: post #3

When I see anything that touches web coding practices for Go I always look up an area that I know best: sanitization. I wrote https://github.com/microcosm-cc/bluemonday which is a pure Go HTML sanitizer inspired by https://github.com/owasp/java-html-sanitizer . The key things to understand about HTML sanitizers: * They must be whitelist based * They must be aware of context * You must sanitize ALL user input even if…

Nice. I use bluemonday, and I also took a lot of inspiration from how microcosm was written. Just wanted to say thanks for the effort!

Re: Go Language – Web Application Secure Coding Practices

#39

Post by paulddraper »

taeric wrote: post #35
osteele wrote: post #13

Earlier quoted context omitted.

> You must sanitize ALL user input even if you don't think you're going to render it on a web page. I'm not able to make sense of this. Sanitize it for what context? SQL? JSON? HTML? Inclusion as a command-line argument? All of these, and hope that sanitizing it for one context doesn't un-sanitize it for others?

I've always had issue with this advice for this very reason. Worse, when people sanitize in and out of the database. Double encoded html just cracks me up. I'm a big fan of not necessarily sanitizing, but treating it appropriate in context. This may mean removing characters, or mapping them, or just delimiting the entire thing. To that end, I argue that you should not necessarily sanitize on the way to the storage me…

Right. It should be SQL sanitized (parameterized) going into the database, and and then HTML/PDF/whatever Santosh when it is put in one of those formats.

Re: Go Language – Web Application Secure Coding Practices

#40

Post by paulddraper »

bascule wrote: post #23
m_sahaf wrote: post #6

Earlier quoted context omitted.

To be fair to the author, the paragraph immediately following that code snippet says: "However, this approach has several flaws and should not be used. It is given here only to illustrate the theory with a practical example. The next section explains how to correctly salt passwords in real life." and goes on to use bcrypt.

To be fair to the person you're responding to, do you think people looking for copypasta are going to check the fine print? The code sample in question is in no way clearly labeled as being problematic, and a tiny statement buried in a large body of text does not change that fact.

To be fair to the person you are responding to, I don't think copypasta developers are going to create secure applications anyway.
Post reply on HN