[dead]
The dangers of single line regular expressions
51–60 of 133 posts
Re: The dangers of single line regular expressions
#52[dead]
Re: The dangers of single line regular expressions
#53Escape the output based on the context a string is being used in versus trying to sanitize for all use cases on input.
This will guarantee that you’re safe no matter how a piece of content is used tomorrow (just need a new escaping function for that content type), and prevent awkward things like not letting users use “unsafe” strings as input. JSX and XHP are example templating systems that understand context and escape appropriately.
If a user wants their title to be “hello%0a%3C%25%3D%20File.open%28%27flag.txt%27%29.read%20%25%3E”, so be it.
Use input validation / parsing to ensure data types aren’t violated, but not as an output safety mechanism.
Re: The dangers of single line regular expressions
#54I pretty much always consider regex expressions as the wrong solution. They're notoriously hard to get right. There's a whole lot of faulty expressions out there for validating email addresses. I prefer to do less validation and let it fail. If the email address is wrong, whatever service you're using for sending emails will just reject it. If you really do need to validate email addresses, use something somebody els…
Here, you have good advice: "I ... consider regex expressions as the wrong solution. They're notoriously hard to get right."
However, the conclusion of "use something somebody else wrote that does it properly", while valid, is asking a lot. As regex is hard to get right, don't assume the code you find on the web or book or via some other means works correctly.
My rule is if I didn't write it and can't wrap my head around the code to convince myself it is the right solution, I don't use it. And as I think others have written, there are some interactive online tests for regex expressions that can help.
Re: The dangers of single line regular expressions
#55I pretty much always consider regex expressions as the wrong solution. They're notoriously hard to get right. There's a whole lot of faulty expressions out there for validating email addresses. I prefer to do less validation and let it fail. If the email address is wrong, whatever service you're using for sending emails will just reject it. If you really do need to validate email addresses, use something somebody els…
Sometimes valid email addresses will be rejected as invalid, and sometimes invalid email addresses are still successfully delivered. Validation guarantees nothing, and at most it should be a UI cue.
Re: The dangers of single line regular expressions
#56Raku (perl6) was a chance for Larry Wall to fix some of the limitations of the perl regex syntax, as you would expect from the perl heritage, it behaves similarly. ~ > raku -e 'say "foobar" ~~ /^ +$/ ?? "yes" !! "no"' yes ~ > raku -e 'say "foo\nbar" ~~ /^ +$/ ?? "yes" !! "no"' no ~ > raku -e 'say "foo\nbar" ~~ /^^ +$$/ ?? "yes" !! "no"' yes - ^^ and $$ are the raku flavour of multiline mode - ~~ the smartmatch operat…
Perl has supported whitespace and comments in regular expressions since approximately forever. Just use the /x modifier. All that Raku did was make that flag a default.
The same thing is available in many other languages. They copied it when they copied from Perl. For example Python's https://docs.python.org/3/library/re.html#flags documents that re.X, also called re.VERBOSE, does the same exact thing.
The fact that people don't use it is because few people care to learn regular expressions well enough to even know that it is an option. One of my favorite examples of astounding people with this was when I was writing a complex stored procedure in PostgreSQL. I read https://www.postgresql.org/docs/current/functions-matching.h.... I looked for flags. And yup, there is an x flag. It turns on "extended syntax". Which does the same exact thing. I needed a complex regular expression that I knew my coworkers couldn't have written themselves. So I commented the heck out of it. They couldn't believe that that was even a thing that you could do!
Re: The dangers of single line regular expressions
#57Seems to me this is more about the danger of passing anything derived from user input into the TEMPLATE side of a templating engine. Why in the world would you ever do that?!?
Obviously if you pass data into the variable side of the engine, you hardly have to worry about it at all, since it's already going into a place that was designed for handling arbitrary and possibly-hostile input and been battle-tested at doing it correctly in Production for many years. If you pass it into the template side, you're betting that you can be as good as dozens of templating engine writers working for a decade at doing that, in exchange for, well, I can't really think of any possible legitimate advantage for doing that.
Re: The dangers of single line regular expressions
#58Seems to me this is more about the danger of passing anything derived from user input into the TEMPLATE side of a templating engine. Why in the world would you ever do that?!? Obviously if you pass data into the variable side of the engine, you hardly have to worry about it at all, since it's already going into a place that was designed for handling arbitrary and possibly-hostile input and been battle-tested at doing…
What if you want to allow users to regex search their documents?
Re: The dangers of single line regular expressions
#59Escape the output based on the context a string is being used in versus trying to sanitize for all use cases on input. This will guarantee that you’re safe no matter how a piece of content is used tomorrow (just need a new escaping function for that content type), and prevent awkward things like not letting users use “unsafe” strings as input. JSX and XHP are example templating systems that understand context and esc…
>If a user wants their title to be “hello%0a%3C%25%3D%20File.open%28%27flag.txt%27%29.read%20%25%3E”, so be it.
that's a good way to horizontally propagate/reflect XSS and other Code As Data vulnerabilities.
better to strip the known-bad/problematic characters
Re: The dangers of single line regular expressions
#60I pretty much always consider regex expressions as the wrong solution. They're notoriously hard to get right. There's a whole lot of faulty expressions out there for validating email addresses. I prefer to do less validation and let it fail. If the email address is wrong, whatever service you're using for sending emails will just reject it. If you really do need to validate email addresses, use something somebody els…
Here, you have good advice: "I ... consider regex expressions as the wrong solution. They're notoriously hard to get right." However, the conclusion of "use something somebody else wrote that does it properly", while valid, is asking a lot. As regex is hard to get right, don't assume the code you find on the web or book or via some other means works correctly. My rule is if I didn't write it and can't wrap my head ar…
I think the average developer has a better chance of finding a robust, battle tested library to do what they need than cooking up some regex of their own. Preferably, the library does not use regex at all and checks data more intelligently.