Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parse, Don't Validate (2019)

lexi-lambda.github.io

Post reply on HN
Print view View original

121–130 of 175 posts

Re: Parse, Don't Validate (2019)

#121

Post by hackrmn »

This article has done rounds on the ITernet before. Maybe because it resonates with people (who repost it time and again). Anyway, I very much agree with the idea. In my experience, "text" or "string" is not a type. Technically it is one, of course, but I seldom see good use of it for when a more apt type would do better -- in short, it's a last resort thing, and it fares badly there too. Ironically, the only good use for it is as input to a... parser.

I see a lot of URLs being passed around as strings within a system perfectly capable of leveraging typing theory and offering user defined types, if not at least through OOP goodness a lot of people would furiously defend. The URL, in this case, would often have _already_ been parsed once, but effectively "unparsed" and keeps being sent around as text in need of parsing at every "junction" of the system that requires to meaningfully access it, except that parsing is approached like some ungodly litany best avoided and thus foregone or lazily implemented with a regex where a regex isn't nearly sufficient. Perhaps it's because we lack parsers, by and large, or in the very least parser generators that are readily available, understandable (to your average developer), and simple enough to use without requiring to understand formal language theory with Chomsky hierarchy, context sensitivity, grammar ambiguity and parse forests, to say the least.

Same with [file] paths, HTTP header values, and other things that seem alluring to dismiss as only being text.

It wouldn't be a problem, had I not seen time and again how the "text" breaks -- URLs with malformed query parameters because why not just do `+ '?' + entries.map(([ name, value ]) => name + "=" + value).join("&")`, how hard can it be? Paths that assume leading slash or lack there of etc.

I believe the article was born precisely of the same class of frustrations. So I am now bringing the same mantra everywhere with me: "There is no such type as string". Parse at earliest opportunity, lazily if the language allows it (most languages do) -- breadth first so as to not pay upfront, just don't let the text slip through.

I am talking from experience, really, your mileage may vary.

Re: Parse, Don't Validate (2019)

#122

Post by exodys »

Maybe I am being contrarian, or maybe I don't understand; if I am reading input, I am always going to validate that input after parsing. Especially if it is from a user.

I understand that they should be separate, but they should be very close together.

Re: Parse, Don't Validate (2019)

#123

Post by mejutoco »

squeaky-clean wrote: post #117
mejutoco wrote: post #111

Earlier quoted context omitted.

My preferred solution would be: You have 2 types UnvalidatedEmail ValidatedEmail Then ValidatedEmail is only created in the function that does the validation: a function that takes an UnvalidatedEmail and returns a ValidatedEmail or an error object.

That can work in some situations. One thing I won't like about it in some other situations is that you now have 2 nullable fields associated with your user, or whatever that email is associated with. It's annoying or even impossible in a lot of systems to have a guaranteed validation that user.UnvalidatedEmail or user.ValidatedEmail must exist but not both.

I see. In my example they would be just types and internally a newtype string.

So an object could have a field

email: UnvalidatedEmail | ValidatedEmail

Nothing would be nullable there in that case. You could match on the type and break if not all cases are handled.

Re: Parse, Don't Validate (2019)

#124

Post by squeaky-clean »

lock1 wrote: post #96
squeaky-clean wrote: post #64

Earlier quoted context omitted.

The easiest and most robust way to deal with email is to have 2 fields. string email, bool isValidated. (And you'll need some additional way to handle a time based validation code). Accept the user's string, fire off an email to it and require them to click a validation link or enter a code somewhere. Email is weird and ultimately the only decider of a valid email is "can I send email to this address and get confirma…

In the spirit of "Parse, Don't Validate", rather than encode "validation" information as a boolean to be checked at runtime, you can define `Email { raw: String }` and hide the constructor behind a "factory function" that accepts any string but returns `Option ` or `Result `. If you need a stronger guarantee than just a "string that passes simple email regex", create another "newtype" that parses the `Email` type fur…

I would still usually prefer email as just a string and validation as a separate property, and they both belong to some other object. Unless you really only want to know if XYZ email exists, it's usually something more like "has it been validated that ABC user can receive email at XYZ address".

Is the user account validated? Send an email to their email string. Is it not validated? Then why are we even at a point in the code where we're considering emailing the user, except to validate the email.

You can use similar logic to what you described, but instead with something like User and ValidatedUser. I just don't think there's much benefit to doing it with specifically the email field and turning email into an object. Because in those examples you can have a User whose email property is a ParseError and you still end up having to check "is the email property result for this user type Email or type ParseError?" and it's very similar to just checking a validation bool except it's hiding what's actually going on.

Re: Parse, Don't Validate (2019)

#126

Post by thom »

seanwilson wrote: post #2

Maybe I'm missing something and I'm glad this idea resonates, but it feels like sometime after Java got popular and dynamic languages got a lot of mindshare, a large chunk of the collective programming community forgot why strong static type checking was invented and are now having to rediscover this. In most strong statically typed languages, you wouldn't often pass strings and generic dictionaries around. You'd nat…

My experience was that enterprise programmers burned out on things like WSDL at about the same time Rails became usable (or Django if you’re that way inclined). Rails had an excellent story for validating models which formed the basis for everything that followed, even in languages with static types - ASP.NET MVC was an attempt to win Rails programmers back without feeling too enterprisey. So you had these very convenient, very frameworky solutions that maybe looked like you were leaning on the type system but really it was all just reflection. That became the standard in every language, and nobody needed to remember “parse don’t validate” because heavy frameworks did the work. And why not? Very few error or result types in fancy typed languages are actually suited for showing multiple (internationalised) validation errors on a web page.

The bitter lesson of programming languages is that whatever clever, fast, safe, low-level features a language has, someone will come along and create a more productive framework in a much worse language.

Note, this framework - perhaps the very last one - is now ‘AI’.

Re: Parse, Don't Validate (2019)

#127

Post by zahlman »

esafak wrote: post #82
guerrilla wrote: post #58

Earlier quoted context omitted.

> I think you have a very rose-tinted view of the past I think they also forgot the entire Perl era.

That's understandable. Youthful indiscretion is best forgotten.

I can still remember trying to deal with structured binary data in Perl, just because I didn't want to fiddle around with memory management in C. I'm not sure it was actually any less painful, and I ultimately abandoned that first attempt.

(Decades later, my "magnum opus" has been through multiple mental redesigns and unsatisfactory partial implementations. This time, for sure...)

Re: Parse, Don't Validate (2019)

#128

Post by Jtsummers »

exodys wrote: post #122

Maybe I am being contrarian, or maybe I don't understand; if I am reading input, I am always going to validate that input after parsing. Especially if it is from a user. I understand that they should be separate, but they should be very close together.

> if I am reading input, I am always going to validate that input after parsing.

In the "parse, don't validate" mindset, your parsing step is validation but it produces something that doesn't require further validation. To stick with the non-empty list example, your parse step would be something like:

  parse [h|t] = Just h :| t
  parse []    = Nothing
So when you run this you can assume that the data is valid in the rest of the code (sorry, my Haskell is rusty so this is a sketch, not actual code):

  process data =
    do {
      Just valid
That has performed validation, but by parsing it also produces a value that doesn't require any revalidation. Every function that takes the parsed data as an argument can ignore the possibility that the data is invalid. If all you do is validate (returning true/false):

  validate [h|t] = true
  validate []    = false
Then you don't have that same guarantee. You don't know that, in future uses, that the data is actually valid. So your code becomes more complex and error-prone.

  process data =
    if validate data then use data else fail "Well shit"

  use [h|t] = do_something_with h t
  use []    = fail "This shouldn't have happened, we validated it right? Must have been called without data being validated first."
The parse approach adds a guarantee to your code, that when you reach `use` (or whatever other functions) with parsed and validated data that you don't have to test that property again. The validate approach does not provide this guarantee, because you cannot guarantee that `use` is never called without first running the validation. There is no information in the program itself saying that `use` must be called after validation (and that validation must return true). Whereas a version of `use` expecting NonEmpty cannot be called without at least validating that particular property.

Re: Parse, Don't Validate (2019)

#129

Post by xboxnolifes »

dylan604 wrote: post #26
jalk wrote: post #16

Earlier quoted context omitted.

An explicit type

Obviously the pseudo code leaves to the imagination, but what benefits does this give you? Are you checking that it is 10-digits? Are you allowing for + symbols for the international codes?

You have functions

    void callNumber(string phoneNumber);
    void associatePhoneNumber(string phoneNumber, Person person);
    Person lookupPerson(string phoneNumber);
    Provider getProvider(string phoneNumber);
I pass in "555;324+289G". Are you putting validation logic into all of those functions? You could have a validation function you write once and call in all of those functions, but why? Why not just parse the phone number into an already validated type and pass that around?

    PhoneNumber PhoneNumber(string phoneNumber);
    void callNumber(PhoneNumber phoneNumber);
    void associatePhoneNumber(PhoneNumber phoneNumber, Person person);
    Person lookupPerson(PhoneNumber phoneNumber);
    Provider getProvider(PhoneNumber phoneNumber);
Put all of the validation logic into the type conversion function. Now you only need to validate once from string to PhoneNumber, and you can safely assume it's valid everywhere else.

Re: Parse, Don't Validate (2019)

#130

Post by kstrauser »

waffletower wrote: post #90

I'm sorry, I don't like to title drop, but I am a Staff Data Engineer and I find that "type driven" development is an inappropriate world view for many programming contexts that I encounter. I use "world view" carefully as it makes a contractual assumption about reality -- "give me what I expect". Data processing does not always have the luxury of such imposition. In these contexts a dynamic and introspective world v…

To amplify what yakshaving said, this may be the worst forum in the entire industry to title drop in. Half the people in any given article's comments are a CxO or Chief or Head or Director or Founder or whatever, or wrote the article, or invented the technology in the article, or are otherwise renowned for something or another.

See also: "Did you win the Putnam?"

Post reply on HN