[dead]
The dangers of single line regular expressions
111–120 of 133 posts
Re: The dangers of single line regular expressions
#112[dead]
Re: The dangers of single line regular expressions
#113Earlier quoted context omitted.
Still seems like that is broken. Shouldn't they be escaping whatever control characters? Like if your user wanted to highlight "Now 75% off". Seems like it is reasonable to want to allow that.
That's a completely different problem: it may be too closed. But it's definitely not a fail open system. It's a fail closed system with a bug.
Yeah but the real bug is the trying to roll-your-own, instead of using a `ERB.escape_tainted_input` method or somesuch. Either that method doesn't exist, which seems like major mis-feature, or the author didn't know about it, or didn't want to use it.
Re: The dangers of single line regular expressions
#114Earlier quoted context omitted.
Still seems like that is broken. Shouldn't they be escaping whatever control characters? Like if your user wanted to highlight "Now 75% off". Seems like it is reasonable to want to allow that.
That's a completely different problem: it may be too closed. But it's definitely not a fail open system. It's a fail closed system with a bug.
Whoops you are absolutely right!! Good point, I totally misread the if/else.
Re: The dangers of single line regular expressions
#115Earlier quoted context omitted.
>If a user wants their title to be “hello%0a%3C%25%3D%20File.open%28%27flag.txt%27%29.read%20%25%3E”, so be it. that's a good way to horizontally propagate/reflect XSS and other Code As Data vulnerabilities. better to strip the known-bad/problematic characters https://en.wikipedia.org/wiki/Code_as_data
And that's how you end up pissing off users with apostrophes in their names.
one apostrophe? sure. more than two in a row? no.
Ku' 'Laangah't is valid.
Re: The dangers of single line regular expressions
#116In my experience `$` does reliably mean end of string for regular expressions, unless you specifically ask for "multiline" mode. Ruby seems to be in multiline mode all the time? $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foobar") else "no"' yes $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\nbar") else "no"' no $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\…
Note that Ruby also has \z which is what you generally want instead of \Z. (\Z allows a trailing newline, \z does not)
You want \Z in Python, and \z in most other languages, to match on end of string. But in some languages $ really does match end of string. As always, you must check your docs.
Re: The dangers of single line regular expressions
#117Escape the output based on the context a string is being used in versus trying to sanitize for all use cases on input. This will guarantee that you’re safe no matter how a piece of content is used tomorrow (just need a new escaping function for that content type), and prevent awkward things like not letting users use “unsafe” strings as input. JSX and XHP are example templating systems that understand context and esc…
The output is not the problem here, it is the input. And, if you can get away with, accepting a small set of known-safe characters is much safer than accepting any character and hoping it will be properly escaped at every level. When the user hands you a string and you then pass this down to other bits of code, you can't know if it will be used in an SQL query, a regex, in an error message that will be rendered into…
What was actually invalid about that input? Why shouldn’t that html escaped string be shown as is to the user?
I guess I would tweak my first comment and say input filtering is not enough. You must do output filtering to truly be safe.
Re: The dangers of single line regular expressions
#118In my experience `$` does reliably mean end of string for regular expressions, unless you specifically ask for "multiline" mode. Ruby seems to be in multiline mode all the time? $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foobar") else "no"' yes $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\nbar") else "no"' no $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\…
False. "$" does NOT mean end-of-string in Perl, Python, PHP, Ruby, Java, or .NET. In particular, a trailing newline (at least) is accepted in those languages. A $ does mean end-of-string in Javascript, POSIX, Rust (if using its usual package), and Go. I'm working with the OpenSSF best practices working group to create some guidance on this stuff. It's a very common misconception. Stay tuned. If anyone knows of vulner…
$ php -v
PHP 8.2.7 (cli) (built: Jun 9 2023 19:37:27) (NTS)
$ php -r 'var_dump(preg_match("/^[a-z0-9 ]+\$/", "hello world"));'
int(1)
$ php -r 'var_dump(preg_match("/^[a-z0-9 ]+\$/", "hello\nworld"));'
int(0)
$ php -r 'var_dump("hello\nworld");'
string(11) "hello
world"
...
$ php -v
PHP 7.2.26-1+0~20191218.33+debian8~1.gbpb5a34b (cli) (built: Dec 18 2019 16:09:52) ( NTS )
$ php -r 'var_dump(preg_match("/^[a-z0-9 ]+\$/", "hello world"));'
int(1)
$ php -r 'var_dump(preg_match("/^[a-z0-9 ]+\$/", "hello\nworld"));'
int(0)
$ php -r 'var_dump("hello\nworld");'
string(11) "hello
world"
edit: https://3v4l.org/N4o8D suggests that the behavior here is identical for all versions of PHP from 4.3 to 8.3.6.
Re: The dangers of single line regular expressions
#119In my experience `$` does reliably mean end of string for regular expressions, unless you specifically ask for "multiline" mode. Ruby seems to be in multiline mode all the time? $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foobar") else "no"' yes $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\nbar") else "no"' no $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\…
False. "$" does NOT mean end-of-string in Perl, Python, PHP, Ruby, Java, or .NET. In particular, a trailing newline (at least) is accepted in those languages. A $ does mean end-of-string in Javascript, POSIX, Rust (if using its usual package), and Go. I'm working with the OpenSSF best practices working group to create some guidance on this stuff. It's a very common misconception. Stay tuned. If anyone knows of vulner…
yes that's correct. I came from perl and python, and got caught out a few times in Go(lang).
Re: The dangers of single line regular expressions
#120In my experience `$` does reliably mean end of string for regular expressions, unless you specifically ask for "multiline" mode. Ruby seems to be in multiline mode all the time? $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foobar") else "no"' yes $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\nbar") else "no"' no $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\…
False. "$" does NOT mean end-of-string in Perl, Python, PHP, Ruby, Java, or .NET. In particular, a trailing newline (at least) is accepted in those languages. A $ does mean end-of-string in Javascript, POSIX, Rust (if using its usual package), and Go. I'm working with the OpenSSF best practices working group to create some guidance on this stuff. It's a very common misconception. Stay tuned. If anyone knows of vulner…
So, you are technically correct when you say PHP accepts a trailing newline, but it doesn't mean it refutes the comment and the context we are discussing.
This is easily demonstrated with an example.