Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parse, Don't Validate (2019)

lexi-lambda.github.io

Post reply on HN
Print view View original

171–180 of 288 posts

Re: Parse, Don't Validate (2019)

#171

Post by kortex »

jimmaswell wrote: post #119
kortex wrote: post #84

This principle is how pydantic[0] utterly revolutionized my python development experience. I went from constantly having to test functions in repls, writing tons of validation boilerplate, and still getting TypeErrors and NoneTypeErrors and AttributeErrors left and right to like...just writing code. And it working ! Like one time I wrote a few hundred lines of python over the course of a day and then just ran it... a…

I've found this to be simply a matter of experience, not tooling. As the years go by I find the majority of my code just working right - never touched anything like pydantic or validation boilerplate for my own code, besides having to write unit tests as an afterthought at work to keep the coverage metric up.

No this was like over a week, and 100% due to the tooling. Pydantic, pycharm, black, mypy, and flake8. Pretty much went from "type hints here and there" to "what happens if I try writing python as if it were (95%) statically typed." I'd been testing well before this point but it's not the same as writing test.

The development process is totally different when you write structured types first and then write your logic. 10/10 would recommend.

Usual caveat: this is what makes sense to me and my brain. Your experience may be different based on neurotype.

Re: Parse, Don't Validate (2019)

#172

Post by kortex »

mixmastamyk wrote: post #137
vikingcaffiene wrote: post #127

Earlier quoted context omitted.

Man, for a dev with as much experience as you’re claiming to have, this comment ain’t a great look. I’d argue that the more experience you get the more you write code for other people which involves adding lots of tooling, tests, etc. Even if the code works the first time, a more senior dev will make sure others have a “pit of success” they can fall into. This involves a lot more than just some “unit tests as an afte…

Adding lots, no. I agree with the grandparent. Keeping the code simple, finding the right abstractions, untangling coupling, gets the most bang for the buck. See the “beyond pep8” talk for a enlightened perspective. That said, lightweight testing and tools like pyflakes to prevent egregious errors helps an experienced dev write very productively. Typing helps the most with large, venerable projects with numerous devs…

> Typing helps the most with large, venerable projects

I disagree. I've started using types from the ground up and it helps almost equally at every stage of the game. Also I aggressively rely on autocomplete for methods. It's faster this way than usual "dynamic" or "pythonic" python.

Part of it might be exactly because writing my datatypes first helps me think about the right abstractions.

The big win with python is maybe 2-10% of functions, I just want to punt and use a dict. But I have shifted >80% of what used to be dicts to Models/dataclasses and it's so much faster to write and easier to debug.

Re: Parse, Don't Validate (2019)

#173

Post by justinpombrio »

ukj wrote: post #170
justinpombrio wrote: post #168

Earlier quoted context omitted.

All those statements are correct. The people downvoting you know that too. I don't think anyone has figured out what point you're trying to make, though. Could you spell it out in more detail? Consider addition. The compiler does type checking, and the JVM actually adds the numbers. Nonetheless, the addition is type checked, and does not represent a weakness of static type checking. How is dynamic dispatch different…

The point is trivial. You can’t have both static type safety AND dynamic dispatch at the same time and in the same context about the same data. Choose one. Give up the other. Make a conscious trade off. The language that you are using is making such trade offs for you - they are implicit in the language design. Best you know what they are because they are meaningful in principle and in practice.

Java has both static type safety AND dynamic dispatch at the same time and in the same context about the same data.

Re: Parse, Don't Validate (2019)

#174

Post by ukj »

justinpombrio wrote: post #173
ukj wrote: post #170

Earlier quoted context omitted.

The point is trivial. You can’t have both static type safety AND dynamic dispatch at the same time and in the same context about the same data. Choose one. Give up the other. Make a conscious trade off. The language that you are using is making such trade offs for you - they are implicit in the language design. Best you know what they are because they are meaningful in principle and in practice.

Java has both static type safety AND dynamic dispatch at the same time and in the same context about the same data.

No, it doesn’t.

The input-data to the compiler can’t be handled by the JVM and vice versa.

The JVM handles byte code as input. The compiler handles source code as input.

That is two different functions with two different data domains.

They literally have different types!

Which one of the two functions is the thing you call “Java”?

Re: Parse, Don't Validate (2019)

#175

Post by DougBTX »

touisteur wrote: post #103
benrbray wrote: post #55

Earlier quoted context omitted.

Yeah, I remember I used to get frustrated when I had to read code that used map() or even .forEach() extensively, thinking a simple, imperative for loop would suffice. I slowly came to realize that a for loop gives you too much power. It's a hammer. It holds the place of a bug you just haven't written yet. Now I'm the one writing JavaScript like it's Haskell. Although Haskell could learn a thing or two from TypeScrip…

On the other end I'm endlessly tired of 'too simple' foreach/map iterators. They're OK until you want to do something like different execution on first and/or last element... Give me a way to implement a 'join' pattern over the foreach iterators, or less terse iterators (with 'some' positional information). I think I'm just ranting about the for-of iterator in Ada...

I quite like the “enumerate” pattern. When indexes matter, instead of `for x in v` you would write, `for (i, x) in enumerate(v)`, then the language only needs one type of for loop as both cases use the same enumerator interface.

Re: Parse, Don't Validate (2019)

#176

Post by justinpombrio »

ukj wrote: post #154
jsnell wrote: post #93

Earlier quoted context omitted.

I think you might need to define what you mean by dynamic dispatch, because it is very clearly something totally different than how the term is commonly understood.

Deciding which implementation of a function handles any given piece of data at runtime. Trivially, because you don’t have this knowledge (and therefore you can’t encode it into your type system) at compile time.

Aha! I think I have debugged your thinking. Wow you made that hard by arguing so much.

Apparently you do know what dynamic dispatch is, you're just wrong that it can't be type checked.

In Java, say you have an interface called `Foo` with a method `String foo()`, and two classes A and B that implement that method. Then you can write this code (apologies if the syntax isn't quite right, it's been a while since I've written Java):

    Foo foo = null;
    if (random_boolean()) {
        foo = new A();
    } else {
        foo = new B();
    }
    // This uses dynamic dispatch
    System.out.println(foo.foo())
This uses dynamic dispatch, but it is statically type checked. If you change A's `foo()` method to return an integer instead of a String, while still declaring that A implements the Foo interface, you will get a type error, at compile time.

Re: Parse, Don't Validate (2019)

#177

Post by hardwaregeek »

vikingcaffiene wrote: post #127
jimmaswell wrote: post #119

Earlier quoted context omitted.

I've found this to be simply a matter of experience, not tooling. As the years go by I find the majority of my code just working right - never touched anything like pydantic or validation boilerplate for my own code, besides having to write unit tests as an afterthought at work to keep the coverage metric up.

Man, for a dev with as much experience as you’re claiming to have, this comment ain’t a great look. I’d argue that the more experience you get the more you write code for other people which involves adding lots of tooling, tests, etc. Even if the code works the first time, a more senior dev will make sure others have a “pit of success” they can fall into. This involves a lot more than just some “unit tests as an afte…

Agreed. It's like saying "oh well I just fly the airplane really carefully". A lot of codebases eclipse the point where one person can understand the whole system. Testing, static analysis and tooling are what allows us to keep the plane flying.

Re: Parse, Don't Validate (2019)

#178

Post by jimmaswell »

JPKab wrote: post #150
vikingcaffiene wrote: post #127

Earlier quoted context omitted.

Man, for a dev with as much experience as you’re claiming to have, this comment ain’t a great look. I’d argue that the more experience you get the more you write code for other people which involves adding lots of tooling, tests, etc. Even if the code works the first time, a more senior dev will make sure others have a “pit of success” they can fall into. This involves a lot more than just some “unit tests as an afte…

It's an immediate tell when someone makes statements like the one you're replying to. It immediately tells me that they've never worked on large software projects, and if they have they haven't worked on ones that lasted more than a few months. I apologize to folks reading this for my rather aggressive tone but I've been writing software for a long time in numerous languages, and people with the unit tests as an afte…

I've worked on large scale projects for a long time. A large portion of the kind of code I've written is impractical or impossible to actually "unit test" e.g. Unity3D components or frontend JS that interacts with a million things. When something weird is going on I'll have to dig in with console logs and breakpoints.

On certain backend code where I am able to do unit tests, they do catch the occasional edge case logic error but not at a rate that makes me concerned about only checking them in some time after the original code, which I'll have already tested myself in real use as I went along.

Re: Parse, Don't Validate (2019)

#179

Post by hardwaregeek »

jolux wrote: post #161
mixmastamyk wrote: post #137

Earlier quoted context omitted.

Adding lots, no. I agree with the grandparent. Keeping the code simple, finding the right abstractions, untangling coupling, gets the most bang for the buck. See the “beyond pep8” talk for a enlightened perspective. That said, lightweight testing and tools like pyflakes to prevent egregious errors helps an experienced dev write very productively. Typing helps the most with large, venerable projects with numerous devs…

Typing is just another guardrail, it's not a substitute for finding the right abstractions and keeping things simple.

I agree but guardrails are pretty awesome. And if people were saying "don't use guardrails, just drive properly", I'd ask why they think guardrails and driving properly are mutually exclusive.

Re: Parse, Don't Validate (2019)

#180

Post by justinpombrio »

ukj wrote: post #174
justinpombrio wrote: post #173

Earlier quoted context omitted.

Java has both static type safety AND dynamic dispatch at the same time and in the same context about the same data.

No, it doesn’t. The input-data to the compiler can’t be handled by the JVM and vice versa. The JVM handles byte code as input. The compiler handles source code as input. That is two different functions with two different data domains. They literally have different types! Which one of the two functions is the thing you call “Java”?

A type system is sound when:

    for all expressions e:
      if e type checks with type t, then one of the following holds:
        - e evaluates to a value v of type t; or
        - e does not halt; or
        - e hits an "unavoidable error"
          like division by 0 or null deref
          (what counts as "unavoidable" varies from lang to lang)
Notice anything interesting about this definition? It uses the word "evaluate"! Type soundness is not just a statement about the type checker. It relates type checking to run time (and thus to compilation too). That is, if you muck with Java's runtime or compiler, you can break type soundness, even if you don't change its type system in the slightest.
Post reply on HN