Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop writing CLI validation. Parse it right the first time

hackers.pub

Post reply on HN
Print view View original

71–80 of 169 posts

Re: Stop writing CLI validation. Parse it right the first time

#71

Post by js2 »

MrJohz wrote: post #52
m463 wrote: post #45

This kind of stuff is what makes me appreciate python's argparse. It's a genuine pleasure to use, and I use it often. If you dig a little deeper into it, it does all the type and value validation, file validation, it does required and mutually exclusive args, it does subargs. And it lets you do special cases of just about anything. And of course it does the "normal" stuff like short + long args, boolean args, args th…

Actually, I think argparse falls into the same trap that the author is talking about. You can define lots of invariants in the parser, and say that these two arguments can't be passed together, or that this argument, if specified, requires these arguments to also be specified, etc. But the end result is a namespace with a bunch of key-value pairs on it, and argparse doesn't play well with typing systems like mypy or…

> What I think the author is looking for is a combination of argparse and Pydantic

Not quite that, but https://typer.tiangolo.com/ is fully type driven.

Re: Stop writing CLI validation. Parse it right the first time

#72

Post by yakshaving_jgt »

bigstrat2003 wrote: post #40
jmull wrote: post #9

> Think about it. When you get JSON from an API, you don't just parse it as any and then write a bunch of if-statements. You use something like Zod to parse it directly into the shape you want. Invalid data? The parser rejects it. Done. Isn’t writing code and using zod the same thing? The difference being who wrote the code. Of course, you hope zod is robust, tested, supported, extensible, and has docs so you can und…

Yeah, the "parse, don't validate" advice seems vacuous to me because of this. Someone is doing that validation. I think the advice would perhaps be phrased better as "try to not reimplement popular libraries when you could just use them".

Parsing includes validation.

The point is you don’t check that your string only contains valid characters and then continue passing that string through your system. You parse your string into a narrower type, and none of the rest of your system needs to be programmed defensively.

To describe this advice as “vacuous” says more about you than it does about the author.

Re: Stop writing CLI validation. Parse it right the first time

#73

Post by dwattttt »

8n4vidtmkvmk wrote: post #66
dwattttt wrote: post #47

Earlier quoted context omitted.

Sibling says this with code, but to distil the advice: reflect the result of your validation in the type system. Then instead of validating a loose type & still using the loose type, you're parsing it from a loose type into a strict type. The key point is you never need to look at a loose type and think "I don't need to check this is valid, because it was checked before"; the type system tracks that for you.

Everyone seems hung up on the type system, but I think the validity of the data is the important part. I'd still want to convert strings to ints, trim whitespace, drop extraneous props and all of that jazz even if I was using plain JS without types. I still wouldn't need to check the inputs again because I know it's already been processed, even if the type system can't help me.

The type isn't just there to make it easy to understand when you do it, it's for you a year later when you need to make a change further inside a codebase, far from where it's validated. Or for someone else who's never even seen the validation section of code.

I'm hung up on the type system because it's a great way to convey the validity of the data; it follows the data around as it flows through your program.

I don't (yet) Typescript, but jsdoc and linting give me enough type checking for my needs.

Re: Stop writing CLI validation. Parse it right the first time

#75

Post by dwattttt »

sudahtigabulan wrote: post #54
dwattttt wrote: post #48

Earlier quoted context omitted.

> options that depend on options What would you do for "top level option, which can be modified in two other ways"? (--option | --option-with-flag1 | --option-with-flag2 | --option-with-flag1-and-flag2) would solve invalid representation, but is unwieldy. Something that results in the usage string [--option [--flag1 --flag2]] doesn't seem so bad at that point.

I think I've seen it done like that --option flag1,flag2 (Maybe with another separator, as long as it doesn't need to be escaped.) Another possibility is to make the main option an argument, like the subcommands in git, systemctl, and others: command option --flag1 --flag2 This depends on the specifics, though.

> --option flag1,flag2

Embedding a second parse step that the first parser doesn't deal with is done, but it's a rough compromise.

It feels like the difficulty in dealing with

  [--option [--flag1 --flag2]]
Is more to do with its expression in the language parsed to, than CLI elegance.

Re: Stop writing CLI validation. Parse it right the first time

#76

Post by mark38848 »

12_throw_away wrote: post #30

I like this advice, and yeah, I always try to make illegal states unrepresentable, possibly even to a fault. The problem I run into here is - how do you create good error messages when you do this? If the user has passed you input with multiple problems, how do you build a list of everything that's wrong with it if the parser crashes out halfway through?

Just use optparse-applicative in PureScript. Applicatives are great for this and the library gives it to you for free.

Re: Stop writing CLI validation. Parse it right the first time

#77

Post by bvrmn »

A valid type for server and port should be a single value. Stop parse it separately please.

":3000" -> use port 3000 with a default host.

"some-host" -> use host with a default port.

"some-host:3000" -> you guess it.

It also allows to extend it to other sources/destinations like unix domain sockets and other stuff without cluttering your CLI options.

Also please consider to use DSN or URI to define database configurations. Host, port, dbname, credentials as separate options or environment variables are quite painful to use.

Re: Stop writing CLI validation. Parse it right the first time

#78

Post by bvrmn »

MrJohz wrote: post #52
m463 wrote: post #45

This kind of stuff is what makes me appreciate python's argparse. It's a genuine pleasure to use, and I use it often. If you dig a little deeper into it, it does all the type and value validation, file validation, it does required and mutually exclusive args, it does subargs. And it lets you do special cases of just about anything. And of course it does the "normal" stuff like short + long args, boolean args, args th…

Actually, I think argparse falls into the same trap that the author is talking about. You can define lots of invariants in the parser, and say that these two arguments can't be passed together, or that this argument, if specified, requires these arguments to also be specified, etc. But the end result is a namespace with a bunch of key-value pairs on it, and argparse doesn't play well with typing systems like mypy or…

In general case generating CLI options from app models leads to horrible CLI UX. Opposite is also true. Working with "nice" CLI options as direct app models is horrendous.

You need a boundary to convert nice opts into nice types. Like pydantic models could take argparse namespace and convert it to something manageable.

Re: Stop writing CLI validation. Parse it right the first time

#79

Post by slifin »

So use Clojure Spec or better yet Malli to parse your input data at the edges of your program

Makes sense, I think a lot of developers would want to complect this problem with their runtime type system of choice without considering the set of downsides for the users

Post reply on HN