Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parse, Don't Validate and Type-Driven Design in Rust

harudagondi.space

Post reply on HN
Print view View original

81–89 of 89 posts

Re: Parse, Don't Validate and Type-Driven Design in Rust

#81

Post by qsera »

instig007 wrote: post #80
qsera wrote: post #69

Earlier quoted context omitted.

were you validating during parsing before?

Validating during parsing is still parsing, there's a reason why `Alternative f` exists after all: you have to choose between branches of possibilities and falsehoods. Now consider that there's another kind of validation that happens outside of program boundaries (where broader-than-needed data is being constrained in a callee rather than the calling site) that should've been expresed as `Alternative f` during parsin…

So you are saying that if at a certain point in parsing the only expected terms are 'a', 'b' and 'c', one should not put the corresponding parsed entry in a `char` (after checking it is either of these aka validating), and instead it should be put in some kind of enum type (parsed via `Alternative f`). Right?

Re: Parse, Don't Validate and Type-Driven Design in Rust

#82

Post by instig007 »

qsera wrote: post #81
instig007 wrote: post #80

Earlier quoted context omitted.

Validating during parsing is still parsing, there's a reason why `Alternative f` exists after all: you have to choose between branches of possibilities and falsehoods. Now consider that there's another kind of validation that happens outside of program boundaries (where broader-than-needed data is being constrained in a callee rather than the calling site) that should've been expresed as `Alternative f` during parsin…

So you are saying that if at a certain point in parsing the only expected terms are 'a', 'b' and 'c', one should not put the corresponding parsed entry in a `char` (after checking it is either of these aka validating), and instead it should be put in some kind of enum type (parsed via `Alternative f`). Right?

You put them however you like, be it in a char or a vector of, but the bottom line is that your parsed items are part of the "sanitized" label that allows you to either tuple-unpack or pattern-match (as long as it's near or literally zero-cost) without performing the same validation ever again for the lifetime of the parsed object. The callees that exclusively expect 'a', 'b' and 'c', and for which they perform internal validation step, should be replaced with versions that permit the inputs with sanitized labels only. How you implement the labels depends on the language at hand, in case of Haskell they can be newtypes or labelled GADTs, but the crucial part is: the "validation" word is systematically pushed to the program boundaries, where it's made part of the parsing interface with `Alternative f` and sanitization labels acting on raw data. In other words you collapse validation into a process of parsing where the result value is already being assembled from a sequence of decisions to branch either with one of the possible successful options or with an error.

Re: Parse, Don't Validate and Type-Driven Design in Rust

#83

Post by qsera »

instig007 wrote: post #82
qsera wrote: post #81

Earlier quoted context omitted.

So you are saying that if at a certain point in parsing the only expected terms are 'a', 'b' and 'c', one should not put the corresponding parsed entry in a `char` (after checking it is either of these aka validating), and instead it should be put in some kind of enum type (parsed via `Alternative f`). Right?

You put them however you like, be it in a char or a vector of, but the bottom line is that your parsed items are part of the "sanitized" label that allows you to either tuple-unpack or pattern-match (as long as it's near or literally zero-cost) without performing the same validation ever again for the lifetime of the parsed object. The callees that exclusively expect 'a', 'b' and 'c', and for which they perform inter…

> but the crucial part is: the "validation" word is systematically pushed to the program boundaries

Yea, so again. Isn't that freaking obvious?! That author seem to be experienced in Haskell where this kind of thing is common knowledge and for some reason this seems to be some kind of revelation to them...

Re: Parse, Don't Validate and Type-Driven Design in Rust

#84

Post by yuppiemephisto »

spockz wrote: post #53
Sharlin wrote: post #32

Earlier quoted context omitted.

Even the newtype-based "parse, don't validate" is tremendously useful in practice, though. The big thing is that if you have a bare string, you don't know "where it's been". It doesn't carry with it information whether it's already been validated. Even if a newtype can't provide you full correctness by construction, it's vastly easier to be convinced of the validity of an encapsulated value compared to a naked one. F…

Coming from Haskell, I loved Agda 2 as a dependent type language. Is there any newer or more mainstream language that has added dependent types?

lean 4

Re: Parse, Don't Validate and Type-Driven Design in Rust

#85

Post by instig007 »

qsera wrote: post #83
instig007 wrote: post #82

Earlier quoted context omitted.

You put them however you like, be it in a char or a vector of, but the bottom line is that your parsed items are part of the "sanitized" label that allows you to either tuple-unpack or pattern-match (as long as it's near or literally zero-cost) without performing the same validation ever again for the lifetime of the parsed object. The callees that exclusively expect 'a', 'b' and 'c', and for which they perform inter…

> but the crucial part is: the "validation" word is systematically pushed to the program boundaries Yea, so again. Isn't that freaking obvious?! That author seem to be experienced in Haskell where this kind of thing is common knowledge and for some reason this seems to be some kind of revelation to them...

> Yea, so again. Isn't that freaking obvious?!

apparently not, as I always find snippets of patterns of this kind from my coworkers (and I've worked in many companies, including the ones that require precision for legal compliance):

    def do_business_stuff(data):
        orders = data.get("orders")
        if not orders:
            return
        for order in orders:
            attr = order.get("attr")
            if attr and len(attr)
The industry's awareness baseline is very low, and it's across tech stacks, Haskell is no exception. I've seen stuff people do with Haskell at 9 to 5 when the only thing devs cared about was to carry on (and preferably migrate to Go), and I wasn't impressed at all (compared to pure gems that can be found on Hackage). So in that sense having the article that says "actually parse once, don't validate everywhere" is very useful, as you can keep sending the link over and over again until people either get tired of you or learn the pattern.

Re: Parse, Don't Validate and Type-Driven Design in Rust

#86

Post by qsera »

instig007 wrote: post #85
qsera wrote: post #83

Earlier quoted context omitted.

> but the crucial part is: the "validation" word is systematically pushed to the program boundaries Yea, so again. Isn't that freaking obvious?! That author seem to be experienced in Haskell where this kind of thing is common knowledge and for some reason this seems to be some kind of revelation to them...

> Yea, so again. Isn't that freaking obvious?! apparently not, as I always find snippets of patterns of this kind from my coworkers (and I've worked in many companies, including the ones that require precision for legal compliance): def do_business_stuff(data): orders = data.get("orders") if not orders: return for order in orders: attr = order.get("attr") if attr and len(attr) The industry's awareness baseline is ver…

But in all seriousness devs could be both be aware and indifferent to it at the same time.

And sometimes, if you are not sure about the constraints 100%, it might even be the safe thing to do.

Re: Parse, Don't Validate and Type-Driven Design in Rust

#87

Post by sam0x17 »

IshKebab wrote: post #25
sam0x17 wrote: post #14

btw the “quoth” crate makes it really really easy to implement scannerless parsing in rust for arbitrary syntax, use it on many of my projects

Interesting looking crate. You don't seem to have any examples at all though so I wouldn't say it makes it easy!

Yeah thanks for highlighting this, I should add some examples

Re: Parse, Don't Validate and Type-Driven Design in Rust

#88

Post by eddd-ddde »

fiddlerwoaroof wrote: post #16
strawhatguy wrote: post #11

The alternative is one type, with many functions that can operate on that type. Like how clojure basically uses maps everywhere and the whole standard library allows you to manipulate them in various ways. The main problem with the many type approach is several same it worse similar types, all incompatible.

Yeah, there's something of a tension between the Perlis quote "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures" and Parse, don't validate. The way I've thought about it, though, is that it's possible to design a program well either by encoding your important invariants in your types or in your functions (especially simple functions). In dynamically typed langua…

Do those design practices protect you when you apply a refactor and now you don't know which call sites may be broken now?

Re: Parse, Don't Validate and Type-Driven Design in Rust

#89

Post by fiddlerwoaroof »

eddd-ddde wrote: post #88
fiddlerwoaroof wrote: post #16

Earlier quoted context omitted.

Yeah, there's something of a tension between the Perlis quote "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures" and Parse, don't validate. The way I've thought about it, though, is that it's possible to design a program well either by encoding your important invariants in your types or in your functions (especially simple functions). In dynamically typed langua…

Do those design practices protect you when you apply a refactor and now you don't know which call sites may be broken now?

Yes
Post reply on HN