Earlier quoted context omitted.
Almost every command line tool has runtime dependencies that must be installed on your system. $ ldd /usr/bin/rg linux-vdso.so.1 (0x00007fff45dd7000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000070764e7b1000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000070764e6ca000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000070764de00000) /lib64/ld-linux-x86-64.so.2 (0x000070764e7e6000) The worst is co…
Don't let your dreams be dreams $ wget 'https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-x86_64-unknown-linux-musl.tar.gz' $ tar -xvf 'ripgrep-14.1.1-x86_64-unknown-linux-musl.tar.gz' $ ldd ripgrep-14.1.1-x86_64-unknown-linux-musl/rg ldd (0x7f1dcb927000) $ file ripgrep-14.1.1-x86_64-unknown-linux-musl/rg ripgrep-14.1.1-x86_64-unknown-linux-musl/rg: ELF 64-bit LSB pie executable, x86-64, v…
Stop writing CLI validation. Parse it right the first time
141–150 of 169 posts
Re: Stop writing CLI validation. Parse it right the first time
#142Rust with Clap solved this forever ago. Also - don't write CLI programs in languages that don't compile to native binaries. I don't want to have to drag around your runtime just to execute a command line tool.
And don't write programs with languages that depend on CMake and random tarballs to build and/or shared libraries to run.
I usually have a lot less issues with dragging a runtime than fighting with builds.
Re: Stop writing CLI validation. Parse it right the first time
#143Rust with Clap solved this forever ago. Also - don't write CLI programs in languages that don't compile to native binaries. I don't want to have to drag around your runtime just to execute a command line tool.
Almost every command line tool has runtime dependencies that must be installed on your system. $ ldd /usr/bin/rg linux-vdso.so.1 (0x00007fff45dd7000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000070764e7b1000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000070764e6ca000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000070764de00000) /lib64/ld-linux-x86-64.so.2 (0x000070764e7e6000) The worst is co…
Re: Stop writing CLI validation. Parse it right the first time
#144Rust with Clap solved this forever ago. Also - don't write CLI programs in languages that don't compile to native binaries. I don't want to have to drag around your runtime just to execute a command line tool.
Almost every command line tool has runtime dependencies that must be installed on your system. $ ldd /usr/bin/rg linux-vdso.so.1 (0x00007fff45dd7000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000070764e7b1000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000070764e6ca000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000070764de00000) /lib64/ld-linux-x86-64.so.2 (0x000070764e7e6000) The worst is co…
This is only a problem, when the program USES a symbol that was only introduced in the newer libc. In other words, when the program made a choice to deliberately need that newer symbol.
Re: Stop writing CLI validation. Parse it right the first time
#145> Think about it. When you get JSON from an API, you don't just parse it as any and then write a bunch of if-statements. You use something like Zod to parse it directly into the shape you want. Invalid data? The parser rejects it. Done. Isn’t writing code and using zod the same thing? The difference being who wrote the code. Of course, you hope zod is robust, tested, supported, extensible, and has docs so you can und…
I think the key part, although the author doesn't quite make it explicit, is that (a) the parsing happens all up front, rather than weaving validation and logic together, and (b) the parsing creates a new structure that encodes the invariants of the application, so that the rest of the application no longer needs to check anything. Whether you do that with Zod or manually or whatever isn't important, the important th…
Re: Stop writing CLI validation. Parse it right the first time
#146Earlier quoted context omitted.
I dont disagree with the desire to get a good API like that. I was just pointing out that this was the core of the desire the author had, as 12_throw_away was correctly pointing out that _true_ parsing and making invalid state unrepresentable forces you to error out on the first missmatch, which makes it impossible to raise multiple issues. the only way around that is to allow invalid state during the input phase. zo…
Why does "true" parsing have to error out on the very first problem? It is more than possible (though maybe not easy) to keep parsing and collecting errors as they appear. Zod, as the given example in the post, does it.
Re: Stop writing CLI validation. Parse it right the first time
#147Earlier quoted context omitted.
Don't let your dreams be dreams $ wget 'https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-x86_64-unknown-linux-musl.tar.gz' $ tar -xvf 'ripgrep-14.1.1-x86_64-unknown-linux-musl.tar.gz' $ ldd ripgrep-14.1.1-x86_64-unknown-linux-musl/rg ldd (0x7f1dcb927000) $ file ripgrep-14.1.1-x86_64-unknown-linux-musl/rg ripgrep-14.1.1-x86_64-unknown-linux-musl/rg: ELF 64-bit LSB pie executable, x86-64, v…
Which only works on linux. No other OS allows static binaries, you always need to link to libc for syscalls.
$ pkg install git rust
$ git clone https://github.com/BurntSushi/ripgrep.git
$ cd ripgrep
$ RUSTFLAGS='-C target-feature=+crt-static' cargo build --release
$ ldd target/release/rg
ldd: target/release/rg: not a dynamic ELF executable
$ file target/release/rg
target/release/rg: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 14.3, FreeBSD-style, with debug_info, not strippedRe: Stop writing CLI validation. Parse it right the first time
#148Earlier quoted context omitted.
When I first saw "Parse, don't validate" title, it struck me as a catchy but perhaps unnecessarily clever catchphrase. It's catchy, yes, but it felt too ambiguous to be meaningful for anyone outside of the target audience (Haskellers in this case). That said, I fully agree with the article content itself. It basically just boils down to: When you create a program, eventually you'll need to process & check whether inp…
What is ValidatedData? A subset of the Data that is valid? This makes no sense to me. The way I see it is you use ‘validate’ when the format of the data you are validating is the exact same format you are gonna be working with right after, meaning the return type doesn’t matter. The return type implies transformation – a write operation per se, whereas validation is always a read operation only.
> What is ValidatedData? A subset of the Data that is valid?
Usually, but not necessarily. `validate()` might add some additional information too, for example: `validationTime`.More often than not, in a real case of applying algebraic data type & "Parse, don't validate", it's something like `Option` or `Result`, borrowing Rust's names. `Option` & `Result` expand the possible return values that function can return to cover the possibility of failure in the validation process, but it's independent from possible values that `ValidatedData` itself can contain.
> The way I see it is you use ‘validate’ when the format of the data you are validating is the exact same format you are gonna be working with right after, meaning the return type doesn’t matter.
The main point of "Parse, don't validate" is to distinguish between "machine-level data representation" vs "possible set of values" of a type and utilize this "possible set of values" property.Your "the exact same format" point is correct; oftentimes, the underlying data representation of a type is exactly the same between pre- & post-validation. But more often than not "possible set of values" of `ValidatedData` is a subset of `Data`. These 2 different "possible set of values" are given their own names in the form of a type `Data` and `ValidatedData`.
This distinction is actually very handy because types can be checked automatically by the (nominal) type system. If you make the `ValidatedData` constructor private & the only way to produce is function `ValidatedData validate(Data)`, then in any part of the codebase, there's no way any `ValidatedData` instance is malformed (assuming `validate` doesn't have bugs).
Extra note: I forgot to mention the "Parse, don't validate" article implicitly implies a nominal type system, where 2 objects with equivalent "data representation" doesn't mean it has the same type. This differs from Typescript's structural type system, where as long as the "data representation" is the same, both object are considered to have the same type.
Typescript will happily accept something like this because of structural
type T1 = { x: String };
type T2 = { x: String };
function f(T1): void { ... }
const t2: T2 = { x: "foo" };
f(t2);
While nominal type systems like Haskell or Java will reject such expressions class T1 { String x; }
class T2 { String x; }
void f(T1) { ... }
// f(new T2()); // Compile error: type mismatch
Because of this, the idea of using type as a "possible set of values" probably felt unintuitive to Typescript folks, as everything is just stringly-typed and different type felt synonymous with different "underlying data representation" there.You can simulate this "same structure, but different meaning" concept of nominal type system in Typescript with some hacky workaround with Symbol.
> The return type implies transformation – a write operation per se, whereas validation is always a read operation only
Why does the return type need to imply transformation and why is "validation" here always read-only? No-op function will return the exact same value you give it (in other words, identity transformation), and Java & Javascript procedures never guarantee a read-only operation.Re: Stop writing CLI validation. Parse it right the first time
#149Earlier quoted context omitted.
I dont disagree with the desire to get a good API like that. I was just pointing out that this was the core of the desire the author had, as 12_throw_away was correctly pointing out that _true_ parsing and making invalid state unrepresentable forces you to error out on the first missmatch, which makes it impossible to raise multiple issues. the only way around that is to allow invalid state during the input phase. zo…
I don't know that I understand why parsing necessarily has to error out on the first mismatch. Good parsers will collect errors as they go along. Zod does take in invalid state as input, but that is what a parser does. In this case, the parser is `any -> T` as opposed to `string -> T`, but that's still a parsing operation.
So, having used this thread to rubber-duck about how the principle of "parse-don't-validate" works with the principle of "provide good error messages", I'm arriving at these rules, which are really more about encapsulation than parsing:
1. Encapsulate both parsing and validation in a single function: `parse(RawInput) -> Result`
2. Ideally, `parse` is implemented by a robust parsing/validation library for the type of input that you're dealing with. It will create some intermediate representations that you need not concern yourself with.
3. If there isn't a good parser library for your use case, your implementation of `parse` will necessarily contain intermediate representations of potentially illegal state. This is both fine and unavoidable, just don't let them leak out of your parser.
Re: Stop writing CLI validation. Parse it right the first time
#150Earlier quoted context omitted.
I think the key part, although the author doesn't quite make it explicit, is that (a) the parsing happens all up front, rather than weaving validation and logic together, and (b) the parsing creates a new structure that encodes the invariants of the application, so that the rest of the application no longer needs to check anything. Whether you do that with Zod or manually or whatever isn't important, the important th…
But when you parse all arguments first before throwing error messages, you can create much better error messages, since they can be more holistic. To do that you need to represent the invalid configuration as a type.
Right - and one thing that keeps coming up for me is that, if you want to maintain complex invariants, it's quite natural to express them in terms of the domain object itself (or maybe, ugh, a DTO with the same fields), rather than in terms of input constraints.