Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parse, Don't Validate (2019)

lexi-lambda.github.io

Post reply on HN
Print view View original

131–140 of 175 posts

Re: Parse, Don't Validate (2019)

#131

Post by tialaramex »

lock1 wrote: post #119
tialaramex wrote: post #81

Earlier quoted context omitted.

I understand where you're coming from, but these terms seem fine to me: This is exactly what, for example, Rust's str::parse method is for. The documentation gives the example: let four: u32 = "4".parse().unwrap(); You will so very often have text and want typed information, and parse is exactly how we do that transformation exactly once. Whereas validation is what it looks like when we try to make piecemeal checks l…

Coming from a more "average imperative" background like C and Java, outside of compiler or serde context, I don't think "parse" is a frequently used term there. The idea of "checking values to see whether they fulfill our expectations or not" is often called "validating" there. So I believe the "Parse, Don't Validate" catchphrase means nothing, if not confusing, to most developers. "Does it mean this 'parse' operatio…

The crucial design choice is that you can't get a Doodad by just saying oh, I'm sure this is a Doodad, I will validate later. You have to parse the thing you've got to get a Doodad if that's what you meant, and the parsing can fail because maybe it isn't one.

    let almost_pi: Rational = "22/7".parse().unwrap();
Here the example is my realistic::Rational. The actual Pi isn't a Rational number so we can't represent it, but 22 divided by 7 is a pretty good approximation considering.

I agree that many languages don't provide a nice API for this, but what I don't see (and maybe you have examples) is languages which do provide a nice API but call it validate. To me that naming would make no sense, but if you've got examples I'll look at them.

Re: Parse, Don't Validate (2019)

#132

Post by jiehong »

seanwilson wrote: post #2

Maybe I'm missing something and I'm glad this idea resonates, but it feels like sometime after Java got popular and dynamic languages got a lot of mindshare, a large chunk of the collective programming community forgot why strong static type checking was invented and are now having to rediscover this. In most strong statically typed languages, you wouldn't often pass strings and generic dictionaries around. You'd nat…

And then clojure enters: let’s keep few data structures but with tons of method.

So things stay as maps or arrays all the way through.

Re: Parse, Don't Validate (2019)

#133

Post by hutao »

zdw wrote: post #66
solomonb wrote: post #24

Earlier quoted context omitted.

I disagree. I think the key insight is to carry the proof with you in the structure of the type you 'parse' into.

Could you clarify what you mean by "carry the proof"?

Typed functional programming has the perspective that types are like propositions and their values are proofs of that proposition. For example, the product type A * B encodes logical conjunction, and having a pair with its first element of type A and its second element of type B "proves" the type signature A * B. Similarly, the NonEmpty type encodes the property that at least one element exists. This way, the program is "correct by construction."

This types-are-propositions persoective is called the Curry-Howard correspondence, and it relates to constructive mathematics (wherein all proofs must provide an algorithm for finding a "witness" object satisfying the desired property).

Re: Parse, Don't Validate (2019)

#134

Post by mrkeen »

exodys wrote: post #122

Maybe I am being contrarian, or maybe I don't understand; if I am reading input, I am always going to validate that input after parsing. Especially if it is from a user. I understand that they should be separate, but they should be very close together.

Suppose you're receiving bytes representing a User at the edge of your system. If you put json bytes into your parser and get back a User, then put your User through validation, that means you know there are both 'valid' Users and 'invalid' Users.

Instead, there should simply be no way to construct an invalid User. But this article pushes a little harder than that:

Does your business logic require a User to have exactly one last name, and one-or-more first names? Some people might go as far as having a private-constructor + static-factory-method create(..), which does the validation, e.g.

  class User {
    private List names;
    private User(List names) {..}
    public static User create(List names) throws ValidationException {
       // Check for name rules here
    }
  }
Even though the create(..) method above validates the name rules, you're still left holding a plain old List-of-Strings deeper in the program when it comes time to use them. The name rules were validated and then thrown away! Now do you check them when you go to use them? Maybe?

If you encode your rules into your data-structure, it might look more like:

  class User {
      String lastName;
      NeList firstNames;
      private User(List names) throws ValidationException {..}
  }
If I were doing this for real, I'd probably have some Name rules too (as opposed to a raw String). E.g. only some non-empty collection of utf8 characters which were successfully case-folded or something.

Is this overkill? Do I wind up with too much code by being so pedantic? Well no! If I'm building valid types out of valid types, perhaps the overall validation logic just shrinks. The above class could be demoted to some kind of struct/record, e.g.

  record User(Name lastName, NeList firstNames);
Before I was validating Names inside User, but now I can validate Names inside Name, which seems like a win:

  class Name {
      private String value;
      private Name (String name) throws ValidationException {..}
  }

Re: Parse, Don't Validate (2019)

#135

Post by exodys »

Jtsummers wrote: post #128
exodys wrote: post #122

Maybe I am being contrarian, or maybe I don't understand; if I am reading input, I am always going to validate that input after parsing. Especially if it is from a user. I understand that they should be separate, but they should be very close together.

> if I am reading input, I am always going to validate that input after parsing. In the "parse, don't validate" mindset, your parsing step is validation but it produces something that doesn't require further validation. To stick with the non-empty list example, your parse step would be something like: parse [h|t] = Just h :| t parse [] = Nothing So when you run this you can assume that the data is valid in the rest o…

Ah, I get it. So, it's just a tagging system. Once tagged, assume valid. DRY.

Re: Parse, Don't Validate (2019)

#136

Post by tialaramex »

mrkeen wrote: post #107
LordDragonfang wrote: post #71

I'll be honest, as someone not familiar with Haskell, one of my main takeaways from this article is going down a rabbit hole of finding out how weird Haskell is. The casualness at which the author states things like "of course, it's obvious to us that `Int -> Void` is impossible" makes me feel like I'm being xkcd 2501'd.

If you spend your life talking about bool having two values, and then need to act as if it has three or 256 values or whatever, that's where the weirdness lives. In C, true doesn't necessarily equal true. In Java (myBool != TRUE) does not imply that (myBool == FALSE). Maybe you could do with some weirdness ! In Haskell: Bool has two members: True & False. (If it's True, it's True. If it's not True, it's False). Unit…

What were you expecting to find? A function which returns an empty type will always diverge - ie there is no return of control, because that return would have a value that we've said never exists. In a systems language like Rust there are functions like this for example std::process::exit is a function which... well, hopefully it's obvious why that doesn't return. You could imagine that likewise if one day the Linux kernel's reboot routine was Rust, that too would never return.

Re: Parse, Don't Validate (2019)

#137

Post by tlavoie »

Along with all the general discussion, I found the concept of defensive parsing striking a chord when reading this as well: "The Seven Turrets of Babel: A Taxonomy of LangSec Errors and How to Expunge Them", https://langsec.org/papers/langsec-cwes-secdev2016.pdf

I'd love for these ideas to take hold at work, but I'm on the fringes in infosec, not a dev.

Re: Parse, Don't Validate (2019)

#138

Post by benhoyt »

I'm not very familiar with functional programming and Haskell in particular. I think I understand the gist of this article, and "use data structures that make illegal states unrepresentable". However, is there a similar article but written with more common languages (C#, C++, Java, Go) in mind? Or is a big part of this concept only relevant for strong functional languages with sum types and pattern matching?

Re: Parse, Don't Validate (2019)

#139

Post by whalesalad »

tialaramex wrote: post #131
lock1 wrote: post #119

Earlier quoted context omitted.

Coming from a more "average imperative" background like C and Java, outside of compiler or serde context, I don't think "parse" is a frequently used term there. The idea of "checking values to see whether they fulfill our expectations or not" is often called "validating" there. So I believe the "Parse, Don't Validate" catchphrase means nothing, if not confusing, to most developers. "Does it mean this 'parse' operatio…

The crucial design choice is that you can't get a Doodad by just saying oh, I'm sure this is a Doodad, I will validate later. You have to parse the thing you've got to get a Doodad if that's what you meant, and the parsing can fail because maybe it isn't one. let almost_pi: Rational = "22/7".parse().unwrap(); Here the example is my realistic::Rational. The actual Pi isn't a Rational number so we can't represent it, b…

The point is parse and validate are interchangeable words for the most part. If you’re parsing something you expect to be an int, but it’s a float or the letter “a” is that not invalid? Is this assessment a form of validating expectations? The line between parsing and validating doesn’t exist.

Re: Parse, Don't Validate (2019)

#140

Post by sn9 »

masklinn wrote: post #62
taylorallred wrote: post #49

Earlier quoted context omitted.

Not speaking for all Go programmers, but I think there is a lot of merit in the idea of "making zero a meaningful value". Zero Is Initialization (ZII) is a whole philosophy that uses this idea. Also, "nil-punning" in Clojure is worth looking at. Basically, if you make "zero" a valid state for all types (the number 0, an empty array, a null pointer) then you can avoid wrapping values in Option types and design your co…

Only if you ignore the billion cases where it doesn't work, such that half the standard library explodes if you try to use it with zero values because they make no sense[0], special mention to reflect.Value's > Panic: call of reflect.Value.IsZero on zero Value And the "cool" stuff like database/sql's plethora of Null* for every single type it can support. So you're not really avoiding "wrapping values in Option types…

I think they're talking about cases when you can make the "zero" behave like an algebraic identity.
Post reply on HN