Of course the only way to be 100% sure would be to actually test it out by sending an actual email isn't it. This is just so obvious. I think the point of email regexes out there is not to make up for the user's silly typing mistake. It's just more to reject nonsensical/malicious/blatantly false inputs etc. as the first layer of protection really, and nobody would really spend tons of time on crafting a "perfect" regex anyways I'm pretty sure, so that's never been a problem at all.
The Correct Way to Validate Email Addresses
391–400 of 405 posts
Re: The Correct Way to Validate Email Addresses
#392Earlier quoted context omitted.
At a small business convincing colleagues isn't that hard. At a larger corporation, getting the rest of the team on-board is the job of whoever is in charge of defining security policies and such. Not allowing characters present on all keyboards and input devices (!"@+'$) statistically increases the risk of people picking weaker passwords then possible, and he/she will guide that change through the proper processes,…
Take that one 'request' - possibly initiated by a jr-mid level developer - and stack it up against the 500 other todo items in the pipeline. You can make those arguments about "statistically increases the risk of people picking weaker passwords " - unless this increases a bottom line or comes out of someone else's budget, this sort of 'bug' is going to be really low down on the totem pole for all the reasons I mentio…
Which gets back to some of the cargo cult thing too. It's not uncommon in a large enterprise to smack up against things like "Years ago we paid a highly trained Security Consultant a large amount of money to develop our Security Guidelines, who are you and where are your security credentials to tell us to do things differently?"
Even worse when that "Security Consultant" is still a retained coworker with a fiefdom to maintain by war at all costs, a wizened old greybeard whose seniority will always trump yours, and/or your boss.
Re: The Correct Way to Validate Email Addresses
#393Earlier quoted context omitted.
> Caring what characters are in the password heavily implies that the site is not hashing the plaintext password in any way, and scarier still, may just be storing the plaintext password as plain text. I don't think that is true at all. I may very well want to put a few simple rules I validate serverside, such as 1) No username in password 2) No email in password 3) No list of 100 most common passwords in password Al…
There's a related issue that I've seen in some banks, which prevents you from using passwords LONGER than a certain number of characters (usually 8 or 10). The only reason I can think this is happening is because they don't hash them and they need to fit in their database column. I took my money out of that bank the next day.
Could just as likely be some process that still needs to pass the password around in a fixed-length file format to some ancient backend process written in COBOL living in the cthulian abyss deep in the bowels of the bank that no one dares rewrite from fear that it would destroy the bank from inside.
Re: The Correct Way to Validate Email Addresses
#394Earlier quoted context omitted.
My guess is that when you call them on the phone, they ask for your password to validate your identity. Which means it's stored in plain text in their database so that customer service can verify what you said is correct. Maybe they don't want their employees to have to be cursed at by customers. I can't think of a good way for a business that has an online interface and frequently handles phone calls from customers…
Someone else in this thread mentioned a company that has customer service type in your password to open your account. So that would be a non-plaintext reason to insist on non-obscene passwords. But it's still terrible, because why the hell is customer service typing in your password. Pretty much all organizations that allow phone authentication seem to be at risk of engineering attacks. The only ones that manage it s…
I'd prefer to have an obscenity in my password if a customer service representative is seeing it. That would help communicate my frustration with their system. Saves me from having to voice that same obscenity, most likely.
Re: The Correct Way to Validate Email Addresses
#395I always assumed it was more a sanitization issue for security's sake. By allowing only a simple subset ("common") email address type, you can be ambivalent about what email server is running and how it reacts to the wide variety of specially crafted email addresses. With no validation other than sending the email, you have to know, for example, what the server would do with an email address that claims to be @localh…
> I always assumed it was more a sanitization issue for security's sake. Sanitization is at best idiotic, at worst creates security problems. There is no such thing as "bad characters", there only is broken code that incorrectly encodes stuff. If you ever find yourself modifying user input "for security reasons" (or really, for any reason at all), you are doing it wrong. The only sane thing to do is to make sure that…
I mostly agree but there are cases where some definition of sanitization is the only appropriate thing. For example, if you allow users to create content with a lightweight subset of HTML for the sake of formatting control and want to render that html in your page. And in such cases, the correct way to sanitize it is not via regexps but via a DOM parser that takes user input and builds a DOM and then emits rendered html according to a whitelist of available tags/attributes. So you might argue DOM parsing isn't sanitization and so still matches your assertion, however, in general it's common and not really inaccurate to call this sanitization.
Re: The Correct Way to Validate Email Addresses
#396Earlier quoted context omitted.
> I always assumed it was more a sanitization issue for security's sake. Sanitization is at best idiotic, at worst creates security problems. There is no such thing as "bad characters", there only is broken code that incorrectly encodes stuff. If you ever find yourself modifying user input "for security reasons" (or really, for any reason at all), you are doing it wrong. The only sane thing to do is to make sure that…
I mostly agree but there are cases where some definition of sanitization is the only appropriate thing. For example, if you allow users to create content with a lightweight subset of HTML for the sake of formatting control and want to render that html in your page. And in such cases, the correct way to sanitize it is not via regexps but via a DOM parser that takes user input and builds a DOM and then emits rendered h…
Well, it depends ... ;-)
The important thing is to not change information. "Sanitization" as it is commonly used means doing something that (potentially) changes information. Which is in contrast to decoding/encoding/parsing/unparsing/translation/..., which, if done correctly, change representation, but not information.
So, to make it a useful distinction, I would call anything that potentially changes the semantics of the processed data "sanitization", and avoid using the term for anything else.
So, simply parsing a string with an HTML parser, possibly checking for acceptable elements, and then serializing back into some sort of canonical form that is semantically equivalent to the input, that's perfectly fine, and I wouldn't call that sanitization, but rather validation and canonicalization.
If you simply start dropping elements, though, that's probably a bad idea, just as simply dropping "Now, it is not always obvious which level of abstraction to evaluate the semantics (and thus the preservation of semantics) at. So, it might be prefectly fine, for example, to remove or replace some elements where the semantics are known and you can show that, say, removing emphasis still generally preserves the meaning of a text.
But a whitelist approach where you simply remove everything that isn't on the whitelist usually is a bad idea. If you want to have a whitelist, use it for validation, and reject anything that's not acceptable, so the user can transform their input in such a way as to avoid any constructs you don't want, while still retaining the meaning of what they are trying to say.
Re: The Correct Way to Validate Email Addresses
#397Earlier quoted context omitted.
I mostly agree but there are cases where some definition of sanitization is the only appropriate thing. For example, if you allow users to create content with a lightweight subset of HTML for the sake of formatting control and want to render that html in your page. And in such cases, the correct way to sanitize it is not via regexps but via a DOM parser that takes user input and builds a DOM and then emits rendered h…
Well, it depends ... ;-) The important thing is to not change information. "Sanitization" as it is commonly used means doing something that (potentially) changes information. Which is in contrast to decoding/encoding/parsing/unparsing/translation/..., which, if done correctly, change representation, but not information. So, to make it a useful distinction, I would call anything that potentially changes the semantics…
I hear what you're saying and it represents an ideal. But there are circumstances where information really has to be removed. Perhaps because the user is no longer present and it was collected under circumstances that had more liberal validation. Or because you're handing information across a boundary of implementation ownership and can't trust the receiver to handle potentially dangerous information correctly. I agree that sanitization (in the sense of stripping bits out of a user data payload according to some security rules) shouldn't be the first tool in the toolbox, but I would really hesitate to say it's always the wrong thing to do.
Edit: here's a good example. I don't know if they still do this, but when I worked at Yahoo!, they used a modified version of PHP that applied a comprehensive sanitization process to all user inputs. As a frontend coder at Y!, all the information you pulled from request parameters, headers, etc, ran through this validation at the PHP level before your app code got to it. You can then literally splat this information into an html page raw, without any further treatment, and not expose the Y! property you work for to an XSS or other injection vectors. There were ways to obtain the raw input using explicit accessors when needed, and these workarounds were detectable by code monitoring tools and had to be reviewed and approved by security team(s). Overall this worked really quite well, in my opinion. Y! could hire junior frontend devs without deep knowledge of data encoding, security issues, etc etc and rest easy. I think the principle of safe-by-default, even if it means destruction of user input in some cases via aggressive sanitization, is a good principle to apply to a frontend framework.
Re: The Correct Way to Validate Email Addresses
#398Earlier quoted context omitted.
Well, it depends ... ;-) The important thing is to not change information. "Sanitization" as it is commonly used means doing something that (potentially) changes information. Which is in contrast to decoding/encoding/parsing/unparsing/translation/..., which, if done correctly, change representation, but not information. So, to make it a useful distinction, I would call anything that potentially changes the semantics…
I hear what you're saying and it represents an ideal. But there are circumstances where information really has to be removed. Perhaps because the user is no longer present and it was collected under circumstances that had more liberal validation. Or because you're handing information across a boundary of implementation ownership and can't trust the receiver to handle potentially dangerous information correctly. I agr…
re edit:
No, that's just a terrible idea. It might work quite well in the sense that it prevents server security holes. But it makes for terrible usability, and potentially even security problems for the user. The user expects that their input is reproduced correctly, and if it isn't, that can potentially have catastrophic consequences because it might result in silent corruption.
See also: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
If you think that it should be possible to have developers who don't understand this problem and its solution, you might as well argue that it should be possible to have developers who don't know any arithmetic or who are illiterate.
If you want to have some help from the computer in avoiding injection attacks, the solution is a type system that ensures that you cannot accidentally use user input as HTML or SQL, for example, or possibly automates coercion when you need to insert pieces of one language into another.
Re: The Correct Way to Validate Email Addresses
#399Earlier quoted context omitted.
My partner has a hyphenated first and last name. So many systems refuse to accept that, and even regular humans struggle with understanding it!
Had this bug in a system I was working on. Pushed a patch in april.. and there still is an ongoing discussion whether it should be merged or not.
Why is there a discussion? What do the people who don't want to take the patch say? "We don't care if we lose customers with unusual names"?
Re: The Correct Way to Validate Email Addresses
#400Earlier quoted context omitted.
Even Linux tools like GNOME Archive Manager in Linux Mint 18 rejected my RAR password containing a $ as the incorrect password, even though it was the correct password for the RAR file I was trying to extract. I then used the command-line unrar utility with the exact same password, and it extracted successfully. Now, why would you preemptively (and explicitly) throw out a candidate password string based on its charac…
I'm guessing both are because the are calling command line tools. One of my pet peeves with linux is that many of these tools are only callable via text and don't expose an API for other programs.
On a Unix system, a console application is effectively just a vararg function taking a bunch of const char* arguments. There's no limitation on what characters you can pass as those arguments, so that doesn't sound like a valid excuse.
(There are some characters that are treated specially by shells, and require escaping - but you don't normally spawn child processes via a shell.)