Yay for parser combinators in the JS/TS wild!
Stop writing CLI validation. Parse it right the first time
61–70 of 169 posts
Re: Stop writing CLI validation. Parse it right the first time
#62This kind of stuff is what makes me appreciate python's argparse. It's a genuine pleasure to use, and I use it often. If you dig a little deeper into it, it does all the type and value validation, file validation, it does required and mutually exclusive args, it does subargs. And it lets you do special cases of just about anything. And of course it does the "normal" stuff like short + long args, boolean args, args th…
Actually, I think argparse falls into the same trap that the author is talking about. You can define lots of invariants in the parser, and say that these two arguments can't be passed together, or that this argument, if specified, requires these arguments to also be specified, etc. But the end result is a namespace with a bunch of key-value pairs on it, and argparse doesn't play well with typing systems like mypy or…
It's almost like you want compile time type safety
Re: Stop writing CLI validation. Parse it right the first time
#63> Think about it. When you get JSON from an API, you don't just parse it as any and then write a bunch of if-statements. You use something like Zod to parse it directly into the shape you want. Invalid data? The parser rejects it. Done. Isn’t writing code and using zod the same thing? The difference being who wrote the code. Of course, you hope zod is robust, tested, supported, extensible, and has docs so you can und…
Yeah, the "parse, don't validate" advice seems vacuous to me because of this. Someone is doing that validation. I think the advice would perhaps be phrased better as "try to not reimplement popular libraries when you could just use them".
When I first saw "Parse, don't validate" title, it struck me as a catchy but perhaps unnecessarily clever catchphrase. It's catchy, yes, but it felt too ambiguous to be meaningful for anyone outside of the target audience (Haskellers in this case).
That said, I fully agree with the article content itself. It basically just boils down to:
When you create a program, eventually you'll need to process & check whether input data is valid or not. In C-like language, you have 2 options
void validate(struct Data d);
struct ValidatedData;
ValidatedData validate(struct Data d);
It doesn't mean you need to explicitly create or name everything as a "parser". It also doesn't mean "don't validate" either; in `ValidatedData validate(struct Data d)` you'll eventually have "validation" logic similar to the procedure `void` counterpart.
Specifically, the article tries to teach folks to utilize the type system to their advantage. Rather than praying to never forget invoking `validate(d)` on every single call site, make the type signature only accept `ValidatedData` type so the compiler will complain loudly if future maintainers try to shove `Data` type to it. This strategy offloads the mental burden of remembering things from the dev to the compiler.
I'm not exactly sure why the "Parse, don't validate" catchphrase keeps getting reused in other language communities. It's not clear to non-FP community what the distinction between "parser" and "validate", let alone "parser combinator". Yet somehow other articles keep reusing this same catchphrase.
Re: Stop writing CLI validation. Parse it right the first time
#64> Think about it. When you get JSON from an API, you don't just parse it as any and then write a bunch of if-statements. You use something like Zod to parse it directly into the shape you want. Invalid data? The parser rejects it. Done. Isn’t writing code and using zod the same thing? The difference being who wrote the code. Of course, you hope zod is robust, tested, supported, extensible, and has docs so you can und…
I think the key part, although the author doesn't quite make it explicit, is that (a) the parsing happens all up front, rather than weaving validation and logic together, and (b) the parsing creates a new structure that encodes the invariants of the application, so that the rest of the application no longer needs to check anything. Whether you do that with Zod or manually or whatever isn't important, the important th…
The base assumption is parsing upfront cost less than validating along. I thinks it's a common case, but not common enough to apply it as a generic principle.
For instance if validating parameter values requires multiple trips to a DB or other external system, weaving the calls in the logic can spare duplicating these round trips. Light "surface" validation can still be applied, but that's not what we're talking about here I think.
Re: Stop writing CLI validation. Parse it right the first time
#65I like this advice, and yeah, I always try to make illegal states unrepresentable, possibly even to a fault. The problem I run into here is - how do you create good error messages when you do this? If the user has passed you input with multiple problems, how do you build a list of everything that's wrong with it if the parser crashes out halfway through?
I think you're looking at it too literally - what people usually mean with"making invalid state unrepresentable" is in the main application which has your domain code - which should be separate from your inputs He even gives the example of zod, which is a validation library he defines to be a parser. What he wants to say : "I don't want to write my own validation in a CLI, give me a good API already that first valida…
Zod might be a validation library, but it also does type coercion and transforms. I believe that's what the author means by a parser.
Re: Stop writing CLI validation. Parse it right the first time
#66Earlier quoted context omitted.
Yeah, the "parse, don't validate" advice seems vacuous to me because of this. Someone is doing that validation. I think the advice would perhaps be phrased better as "try to not reimplement popular libraries when you could just use them".
Sibling says this with code, but to distil the advice: reflect the result of your validation in the type system. Then instead of validating a loose type & still using the loose type, you're parsing it from a loose type into a strict type. The key point is you never need to look at a loose type and think "I don't need to check this is valid, because it was checked before"; the type system tracks that for you.
Everyone seems hung up on the type system, but I think the validity of the data is the important part. I'd still want to convert strings to ints, trim whitespace, drop extraneous props and all of that jazz even if I was using plain JS without types.
I still wouldn't need to check the inputs again because I know it's already been processed, even if the type system can't help me.
Re: Stop writing CLI validation. Parse it right the first time
#67Re: Stop writing CLI validation. Parse it right the first time
#68Earlier quoted context omitted.
Actually, I think argparse falls into the same trap that the author is talking about. You can define lots of invariants in the parser, and say that these two arguments can't be passed together, or that this argument, if specified, requires these arguments to also be specified, etc. But the end result is a namespace with a bunch of key-value pairs on it, and argparse doesn't play well with typing systems like mypy or…
It's almost like you want compile time type safety
You can have that with Mypy and friends in Python, and Typescript in the JS world. The problem is that older libraries often don't utilise that type safety very well because their API wasn't designed for it.
The library in the original post is essentially a Javascript library, but it's one designed so that if you use it with Typescript, it provides that type safety.
Re: Stop writing CLI validation. Parse it right the first time
#69I like this advice, and yeah, I always try to make illegal states unrepresentable, possibly even to a fault. The problem I run into here is - how do you create good error messages when you do this? If the user has passed you input with multiple problems, how do you build a list of everything that's wrong with it if the parser crashes out halfway through?
Maybe you can use his `or` construct to allow a `--server` without `--port`, but then also add a default `error_message` property.
After parsing you check if `error_message` exists and raise that error.
Re: Stop writing CLI validation. Parse it right the first time
#70Earlier quoted context omitted.
I think the key part, although the author doesn't quite make it explicit, is that (a) the parsing happens all up front, rather than weaving validation and logic together, and (b) the parsing creates a new structure that encodes the invariants of the application, so that the rest of the application no longer needs to check anything. Whether you do that with Zod or manually or whatever isn't important, the important th…
The base assumption is parsing upfront cost less than validating along. I thinks it's a common case, but not common enough to apply it as a generic principle. For instance if validating parameter values requires multiple trips to a DB or other external system, weaving the calls in the logic can spare duplicating these round trips. Light "surface" validation can still be applied, but that's not what we're talking abou…
It's not about costing less, it's about program structure. The goal should be to move from interface type (in this case a series of strings passed on the command line) to internal domain type (where we can use rich data types and enforce invariants like "if server, then all server properties are specified") as quickly as possible. That way, more of the application can be written to use those rich data types, avoiding errors or unnecessary defensive programming.
Even better, that conversion from interface type to internal type should ideally happen at one explicit point in the program - a function call which rejects all invalid inputs and returns a type that enforces the invariants we're interested in. That way, we gave a clean boundary point between the outside world and the inside one.
This isn't a performance issue at all, it's closer to the "imperative shell, functional core" ideas about structuring your application and data.