Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parse, Don't Validate (2019)

lexi-lambda.github.io

Post reply on HN
Print view View original

61–70 of 175 posts

Re: Parse, Don't Validate (2019)

#61

Post by rorylaitila »

I make great use of value objects in my applications but there are things I needed to do to make it ergonomic/performant. A "small" application of mine has over 100 value objects implemented as classes. Large apps easily get into the 1000s of classes just for value objects. That is a lot of boilerplate. It's a lot of boxing/unboxing. It'd be a lot of extra typing than "stringly typed" programs.

To make it viable, all value objects are code-generated from model schemas, and then customized as needed (only like 5% need customization beyond basic data types). I have auto-upcasting on setters so you can code stringly when wanted, but everything is validated (very useful for writing unit tests more quickly). I only parse into types at boundaries or on writes/sets, not on reads/gets (limit's the amount of boxing, particularly on reading large amounts of data). Heavy use of reflection, and auto-wiring/dependency injection.

But with these conventions in place, I quite enjoy it. Easy to customize/narrow a type. One convention for all validation. External inputs are by default secure with nice error messages. Once place where all values validation happens (./values classes folder).

Re: Parse, Don't Validate (2019)

#62

Post by masklinn »

taylorallred wrote: post #49
r4victor wrote: post #36

It seems modern statically-typed and even dynamically-typed languages all adopted this idea, except Go, where they decided zero values represent valid states always (or mostly). A sincere question to Go programmers – what's your take on "Parse, Don't Validate"?

Not speaking for all Go programmers, but I think there is a lot of merit in the idea of "making zero a meaningful value". Zero Is Initialization (ZII) is a whole philosophy that uses this idea. Also, "nil-punning" in Clojure is worth looking at. Basically, if you make "zero" a valid state for all types (the number 0, an empty array, a null pointer) then you can avoid wrapping values in Option types and design your co…

Only if you ignore the billion cases where it doesn't work, such that half the standard library explodes if you try to use it with zero values because they make no sense[0], special mention to reflect.Value's

> Panic: call of reflect.Value.IsZero on zero Value

And the "cool" stuff like database/sql's plethora of Null* for every single type it can support. So you're not really avoiding "wrapping values in Option types", you're instead copy/pasting ad-hoc ones all over, and have to deal with zero values in places where they have no reason to be, forced upon you by the language.

And then of course it looks even worse because... not having universal default values doesn't preclude having opt-in default values. So when that's useful and sensible your type gets a default value, and when it's not it doesn't, and that avoids having to add a check in every single method so your code doesn't go off the rail when it encounters a nonsensical zero value.

[0] or even when you might think it does, like a nil Logger or Handler

Re: Parse, Don't Validate (2019)

#63

Post by ronjakoi »

cogman10 wrote: post #57
FranklinJabar wrote: post #48

Earlier quoted context omitted.

> On your precise exemple, I can even say that I never saw something like an "Email object". Well that's.... absolutely horrifying. Would you mind sharing what industry/stack you work with?

I've seen some devs prefer that route of programming and it very often results in performance problems. An undiscussed issue with "everything is a string or dictionary" is that strings and dictionaries both consume very large amounts of memory. Particularly in a language like java. A java object which has 2 fields in it with an int and a long will spend most of it's memory on the object header. You end up with an obj…

I work with PHP, where classes are supposedly a lot slower than strings and arrays (PHP calls dictionaries "associative arrays").

Re: Parse, Don't Validate (2019)

#64

Post by squeaky-clean »

FranklinJabar wrote: post #48
pjerem wrote: post #7

Earlier quoted context omitted.

> In most strong statically typed languages, you wouldn't often pass strings and generic dictionaries around. In 99% of the projects I worked on my professional life, anything that is coming from an human input is manipulated as a string and most of the time, it stays like this in all of the application layers (with more or less checks in the path). On your precise exemple, I can even say that I never saw something l…

> On your precise exemple, I can even say that I never saw something like an "Email object". Well that's.... absolutely horrifying. Would you mind sharing what industry/stack you work with?

The easiest and most robust way to deal with email is to have 2 fields. string email, bool isValidated. (And you'll need some additional way to handle a time based validation code). Accept the user's string, fire off an email to it and require them to click a validation link or enter a code somewhere.

Email is weird and ultimately the only decider of a valid email is "can I send email to this address and get confirmation of receipt".

If it's a consumer website you can so some clientside validation of ".@.\\..*" to catch easy typos. That will end up rejecting a super small amount of users but they can usually deal with it. Validating against known good email domains and whatnot will just create a mess.

Re: Parse, Don't Validate (2019)

#65

Post by tomtom1337 »

curiousgal wrote: post #9

Semi tangent but I am curious. for those with more experience in python, do you just pass around generic Pandas Dataframes or do you parse each row into an object and write logic that manipulates those instead?

Definitely do not parse each row into eg pydantic models. You lose the entire performance benefit of pandas / polars by doing this.

If you need it, use a dataframe validation library to ensure that values are within certain ranges.

There are not yet good, fast implementations of proper types in Python dataframes (or databases for that matter) that I am aware of.

Re: Parse, Don't Validate (2019)

#66

Post by zdw »

solomonb wrote: post #24
zdw wrote: post #11

This is a great article, but people often trip over the title and draw unusual conclusions. The point of the article is about locality of validation logic in a system. Parsing in this context can be thought as consolidating the logic that makes all structure and validity determination about incoming data into one place in the program. This lets you then rely on the fact that you have valid data in a known structure i…

I disagree. I think the key insight is to carry the proof with you in the structure of the type you 'parse' into.

Could you clarify what you mean by "carry the proof"?

Re: Parse, Don't Validate (2019)

#67

Post by zanecodes »

brooke2k wrote: post #44
seanwilson wrote: post #2

Maybe I'm missing something and I'm glad this idea resonates, but it feels like sometime after Java got popular and dynamic languages got a lot of mindshare, a large chunk of the collective programming community forgot why strong static type checking was invented and are now having to rediscover this. In most strong statically typed languages, you wouldn't often pass strings and generic dictionaries around. You'd nat…

this is very much a nitpick, but I wouldn't call throwing an exception in the constructor a good use of static typing. sure, it's using a separate type, but the guarantees are enforced at runtime

Given that the compiler can't enforce that users only enter valid data at compile time, the next best thing is enforcing that when they do enter invalid data, the program won't produce an `Email` object from it, and thus all `Email` objects and their contents can be assumed to be valid.

Re: Parse, Don't Validate (2019)

#68

Post by mh2266 »

taylorallred wrote: post #49
r4victor wrote: post #36

It seems modern statically-typed and even dynamically-typed languages all adopted this idea, except Go, where they decided zero values represent valid states always (or mostly). A sincere question to Go programmers – what's your take on "Parse, Don't Validate"?

Not speaking for all Go programmers, but I think there is a lot of merit in the idea of "making zero a meaningful value". Zero Is Initialization (ZII) is a whole philosophy that uses this idea. Also, "nil-punning" in Clojure is worth looking at. Basically, if you make "zero" a valid state for all types (the number 0, an empty array, a null pointer) then you can avoid wrapping values in Option types and design your co…

I mean, yeah, you can avoid wrapping things that are optional in Option, but that doesn't make that the result is semantically meaningful. If you have a User struct with "age: 0" does that mean:

1. The user being referred to is a newborn, or:

2. Some other code, locally or on a server you're reading a response from, broke something and forgot to set the age field.

It's not just Rust that gets this (in my opinion) right, Kotlin and Swift also do this well at both a language and serialization library (kotlinx-serialization and Codable) level, but this quote from this article comparing Go and Rust is generally how I think about it:

> If you’re looking to reduce the whole discourse to “X vs Y”, let it be “serde vs crossing your fingers and hoping user input is well-formed”. It is one of the better reductions of the problem: it really is “specifying behavior that should be allowed (and rejecting everything else)” vs “manually checking that everything is fine in a thousand tiny steps”, which inevitably results in missed combinations because the human brain is not designed to hold graphs that big.

https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-...

Basically, if you do not want it to be wrapped in Option then you should either:

1. explicitly specify a default—which can, but doesn't have to be 0.

2. fail immediately (as this article explains) when trying to parse some data that does not have that field set.

and if you do want an Option, you just need to be explicit about that.

Re: Parse, Don't Validate (2019)

#69

Post by dang »

Related. Others?

Parse, Don't Validate (2019) - https://news.ycombinator.com/item?id=41031585 - July 2024 (102 comments)

Parse, don't validate (2019) - https://news.ycombinator.com/item?id=35053118 - March 2023 (219 comments)

Parse, Don't Validate (2019) - https://news.ycombinator.com/item?id=27639890 - June 2021 (270 comments)

Parse, Don’t Validate - https://news.ycombinator.com/item?id=21476261 - Nov 2019 (230 comments)

Parse, Don't Validate - https://news.ycombinator.com/item?id=21471753 - Nov 2019 (4 comments)

Post reply on HN