Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parse, Don't Validate (2019)

lexi-lambda.github.io

Post reply on HN
Print view View original

101–110 of 175 posts

Re: Parse, Don't Validate (2019)

#101

Post by yakshaving_jgt »

waffletower wrote: post #90

I'm sorry, I don't like to title drop, but I am a Staff Data Engineer and I find that "type driven" development is an inappropriate world view for many programming contexts that I encounter. I use "world view" carefully as it makes a contractual assumption about reality -- "give me what I expect". Data processing does not always have the luxury of such imposition. In these contexts a dynamic and introspective world v…

> I don't like to title drop, but I am a Staff Data Engineer

I am a Chief Technology Officer[^1].

Your opinion here is common, and misguided.

Here is why: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...

---

[^1]: Literally nobody cares.

Re: Parse, Don't Validate (2019)

#102

Post by yakshaving_jgt »

danieltanfh95 wrote: post #20

Hot take: Static typing is often touted as the end all be all, and all you need to do is "parse, don't validate" at the edge of your program and everything is fine and dandy. In practice, I find that staunch static typing proponents are often middle or junior engineeers that want to work with an idealised version of programming in their heads. In reality what you are looking for is "openness" and "consistency", becau…

You are yet another person who is misguided in exactly the way described in this article by the same author: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...

Re: Parse, Don't Validate (2019)

#103

Post by Joel_Mckay »

An unconstrained json/bson parser without recursive structure limits must be bounded somehow. In many cases, the ordering of marshaled data cannot be guaranteed across platforms.

The best method is walk the symbolic tree with a cost function, and score the fitness of the data compared to expected structures. For example, mismatched or duplicate GUID/Account/permission/key fields reroute the message to the dead-letter queue for analysis, missing required fields trigger error messaging, and missing optional fields lower the qualitative score of the message content.

Parsers can be extremely unpredictable, and loosely typed formats are dangerous at times. =3

Re: Parse, Don't Validate (2019)

#104

Post by mmis1000 »

This article always end up relevant once in a while.

Recently, I am trying to make llm to output specific format.

It turns out no matter how you wrote propmt and perform validate. It will never be as effective as just limit the output with proper bnf (via llama cpp grammar file).

Re: Parse, Don't Validate (2019)

#105

Post by munificent »

seanwilson wrote: post #2

Maybe I'm missing something and I'm glad this idea resonates, but it feels like sometime after Java got popular and dynamic languages got a lot of mindshare, a large chunk of the collective programming community forgot why strong static type checking was invented and are now having to rediscover this. In most strong statically typed languages, you wouldn't often pass strings and generic dictionaries around. You'd nat…

> You'd naturally gravitate towards parsing/transforming raw data into typed data structures that have guaranteed properties instead to avoid writing defensive code everywhere e.g.

There's nothing natural about this. It's not like we're born knowing good object-oriented design. It's a pattern that has to be learned, and the linked article is one of the well-known pieces that helped a lot of people understand this idea.

Re: Parse, Don't Validate (2019)

#106

Post by zanecodes »

mh2266 wrote: post #95
zanecodes wrote: post #67

Earlier quoted context omitted.

Given that the compiler can't enforce that users only enter valid data at compile time, the next best thing is enforcing that when they do enter invalid data, the program won't produce an `Email` object from it, and thus all `Email` objects and their contents can be assumed to be valid.

This is all pretty language-specific and I think people may end up talking past each other. Like, my preferred alternative is not "return an invalid Email object" but "return a sum type representing either an Email or an Error", because I like languages with sum types and pattern matching and all the cultural aspects those tend to imply. But if you are writing Python or Java, that might look like "throw an exception…

Ah yeah, I guess I assumed by the use of the term "contructor" that GP meant a language like Python or Java, and in some cases it can difficult to prevent misuse by making an unsafe constructor private and only providing a public safe contructor that returns a sum type.

I definitely agree returning a sum type is ideal.

Re: Parse, Don't Validate (2019)

#107

Post by mrkeen »

LordDragonfang wrote: post #71

I'll be honest, as someone not familiar with Haskell, one of my main takeaways from this article is going down a rabbit hole of finding out how weird Haskell is. The casualness at which the author states things like "of course, it's obvious to us that `Int -> Void` is impossible" makes me feel like I'm being xkcd 2501'd.

If you spend your life talking about bool having two values, and then need to act as if it has three or 256 values or whatever, that's where the weirdness lives.

In C, true doesn't necessarily equal true.

In Java (myBool != TRUE) does not imply that (myBool == FALSE).

Maybe you could do with some weirdness!

In Haskell: Bool has two members: True & False. (If it's True, it's True. If it's not True, it's False). Unit has one members: () Void has zero members.

To be fair I'm not sure why Void was raised as an example in the article, and I've never used it. I didn't turn up any useful-looking implementations on hoogle[1] either.

[1] https://hoogle.haskell.org/?hoogle=a+-%3E+Void&scope=set%3As...

Re: Parse, Don't Validate (2019)

#108

Post by munificent »

brooke2k wrote: post #44
seanwilson wrote: post #2

Maybe I'm missing something and I'm glad this idea resonates, but it feels like sometime after Java got popular and dynamic languages got a lot of mindshare, a large chunk of the collective programming community forgot why strong static type checking was invented and are now having to rediscover this. In most strong statically typed languages, you wouldn't often pass strings and generic dictionaries around. You'd nat…

this is very much a nitpick, but I wouldn't call throwing an exception in the constructor a good use of static typing. sure, it's using a separate type, but the guarantees are enforced at runtime

I wouldn't call it a good use of static typing, but I'd call it a good use of object-oriented programming.

This is one of the really key ideas behind OOP that tends to get overlooked. A constructor's job is to produce a semantically valid instance of a class. You do the validation during construction so that the rest of the codebase can safely assume that if it can get its hands on a Foo, it's a valid Foo.

Re: Parse, Don't Validate (2019)

#109

Post by Archelaos »

imtringued wrote: post #73
Archelaos wrote: post #30

Earlier quoted context omitted.

Strong static type checking is helpful when implementing the methodology described in this article, but it is besides its focus. You still need to use the most restrictive type. For example, uint, instead of int, when you want to exclude negative values; a non-empty list type, if your list should not be empty; etc. When the type is more complex, specific contraints should be used. For a real live example: I designed…

Using uint to exclude negative values is one of the most common mistakes, because underflow wrapping is the default instead of saturation. You subtract a big number from a small number and your number suddenly becomes extremely large. This is far worse than e.g. someone having traveled a negative distance.

In C# I use the 'checked' keyword in this or similar cases, when it might be relevant: c = checked(a - b);

Note that this does not violate the "Parse, Don't Validate" rule. This rule does not prevent you from doing stupid things with a "parsed" type.

In other cases, I use its cousin unchecked on int values, when an overflow is okay, such as in calculating an int hash code.

Re: Parse, Don't Validate (2019)

#110

Post by jmull »

d0liver wrote: post #76

I think, more generally, "push effects to the edges" which includes validation effects like reporting errors or crashing the program. If you, hypothetically, kept all of your runtime data in a big blob, but validated its structure right when you created it, then you could pass around that blob as an opaque representation. You could then later deserialize that blob and use it and everything would still be fine -- you'…

Systems tend to change over time (and distributed nodes of a system don’t cut over all at once). So what was valid when you serialized it may not be valid when you deserialize it later.
Post reply on HN