Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python’s “type hints” are a bit of a disappointment to me

uninformativ.de

Post reply on HN
Print view View original

351–360 of 597 posts

Re: Python’s “type hints” are a bit of a disappointment to me

#351

Post by zmgsabst »

ImprobableTruth wrote: post #253
zmgsabst wrote: post #201

Earlier quoted context omitted.

Your suggestion to “use a recursive union type” and that being “ugly … if your type checker doesn’t … support recursive types” being required to support varying JSON is my point: Python types don’t support a common Pythonism used frequently in my work. In fact, you admit that even fixed JSON can be difficult without a 3rd party library. You’re lecturing me like I don’t understand types without realizing that you repe…

I think I might not have been clear: this is not a limitation of the type system , but of a specific type checker (mypy), that will hopefully be fixed soon. Both Pyright and Pyre support recursive type aliases right now . Open one of their playgrounds and you'll see that the following, intuitive definition JSON = Union[ None, bool, int, float, str, List['JSON'], Dict[str, 'JSON'], ] works without issue. >In fact, you…

> this is not a limitation of the type system, but of a specific type checker (mypy), that will hopefully be fixed soon

> Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation.

> it's just the tooling that is lagging a bit behind at the moment

So… the standard library has exactly the problem I articulated, and you’re violently agreeing while speaking down to me.

I think that reflects poorly on you.

Re: Python’s “type hints” are a bit of a disappointment to me

#352

Post by bayesian_horse »

grumpyprole wrote: post #345
bayesian_horse wrote: post #304

Earlier quoted context omitted.

Python libraries aren't representative of Python code. The majority of Python programmers (not "just sysadmins" wherever that slur comes from) never publish or contribute to any libraries. I can compare Python's productivity to other languages, and it beats all of them so far. Another lesson I learned from working with statically typed languages: The quality of the code is more dependent on who writes it than on the…

You might be more productive in Python versus anything else for the scale and complexity of the software you are involved in, but you cannot make sweeping generalisations like this. I also think you fail to understand that in many situations, static type systems actually improve productivity, especially at scale or for non-trivial domains. This is of course why Python is now trying to retrofit them. There's also the…

I "fail to understand" something you actually never did or experienced, right? I know that a lot of companies are maintaining large Python codebases, and the "lack of typechecking" might be one of the biggest pain points, but the total pain is a lot less. That was always my experience and most people who say differently haven't actually used Python that ...

I know that Typescript helped me over Javascript. I don't get that benefit from Mypy yet, but the "pain" just isn't as great as it was with JS. And I've seen what inexperienced programmers do when a static typecheckers try to force or encourage them to do the right thing. It's not necessarily the right thing, just a lot more complicated than it needs to be.

Re: Python’s “type hints” are a bit of a disappointment to me

#353

Post by HelloNurse »

arinlen wrote: post #342
bayesian_horse wrote: post #290

Earlier quoted context omitted.

What Pandas does is notoriously hard to fit into a compile-time type system. Certainly too hard to go into the brains of scientists who didn't grow up coding. No, the code in data science isn't bad because of the lack of typing. The code is "bad" mostly because those writing it are relatively fresh from starting to program. Also there is more pressure to make things possible, often just to run it once, and neglect re…

> What Pandas does is notoriously hard to fit into a compile-time type system. Certainly too hard to go into the brains of scientists who didn't grow up coding. I'm not sure if that's true. Doesn't Pandas handle ETL and some anaysis? There is nothing inherent to ETL that makes it a hard problem with compiled languages. In your opinion, what does Pandas do that's hard to do with compile-time languages?

It isn't a matter of "compile time": explicit type declarations and definitions can often be formally sound but practically worthless.

Significant types in ETL-style applications typically come from outside (e.g. a certain CSV column in the input file contains a date in YYYYMMDD format, or maybe YYYYDDMM, figure it out, and don't forget time zones or your accounting will go wrong).

Then types are mostly complex but obvious and easily deducted (e.g. multiplying matrices of compatible shapes necessarily gives a matrix of a certain shape, why should the program say anything more detailed or lower-level than "do a matrix multiplication" or "do a tensor product"?); they are an often dynamic and unpredictable property of the data, not a useful abstraction.

Re: Python’s “type hints” are a bit of a disappointment to me

#354

Post by wodenokoto »

A lot of people saying author isn't smart/educated enough about the topic, but I kinda agree with him, that they are disappointing. From a "normal" programmers point of view, they are weak and don't really safeguard you well enough out of the box.

Apparently you can install and setup a billion 3rd party libraries in your code editor and a CI pipeline to get a really good workflow, but that just shows that you need to be an expert in order to use these.

Re: Python’s “type hints” are a bit of a disappointment to me

#355

Post by chombier »

neves wrote: post #269
jsmeaton wrote: post #239

Adding type hints to Python has increased my productivity by at least a factor of 10. They allow you to reason about code in a local function without having to track back up dozens of call sites to ensure you're getting what you think you're getting. That alone is worth the price of admission. Both when editing code or reviewing someone else's. It's fantastic, particularly in a very large code base. The editor experi…

Nice. How do you force types just in the new code?

I generally force it on a per-file basis by adding

     # mypy: disallow-untyped-defs
on top of new python files.

Re: Python’s “type hints” are a bit of a disappointment to me

#356

Post by LtWorf_ »

Too wrote: post #296
neves wrote: post #269

Earlier quoted context omitted.

Nice. How do you force types just in the new code?

Mypy can be configured to ignore modules based on their names. Though honestly I think it’s worth it to just bite the bullet and spend a full day or two to go through and fix every single error. After this you will have a much easier time navigating code base. I mean, your “new code” is probably modifications to your old code or calling your old code right? So you want to have at least the boundary between new and ol…

> day or two

Your code must be quite small.

Re: Python’s “type hints” are a bit of a disappointment to me

#357

Post by lars512 »

I love the gradual typing approach, and have found it makes codebases better to maintain, even if some libraries like pandas are a total pain.

My biggest gripe is the conflicting warnings between mypy and pyright. Language servers use pyright by default in the editor, but I strongly believe a Python project shouldn't require nodejs as part of its required CI toolchain. So all my projects use mypy in strict mode in CI. Basically this means that mypy is the standard to meet, and that there are persistent warnings in the editor which I have to ignore.

Re: Python’s “type hints” are a bit of a disappointment to me

#358

Post by t43562 »

arinlen wrote: post #340
t43562 wrote: post #339

Correctness in Python is all about unit tests - if you don't bother with them then you're playing with fire (and of course I often "play with fire"). Type hints .... well they help the IDE to autocomplete which is great. They also help you to get the order of parameters in a function/method call correct which is useful if the parameters aren't all the same type! I converted a medium sized project to type hinting once…

> Correctness in Python is all about unit tests - if you don't bother with them then you're playing with fire (and of course I often "play with fire"). But Python does support type hints, and tools like MyPy do use those type hints to evaluate correctness. It seems to me that Python's type hints are indeed Python's way to check for correctness. Also, unit tests don't verify correctness. They only serve to check whate…

You can have a program with correct type handling that doesn't work but with unit tests you have a program that works and documentation (in the form of the test) about the ways in which the programmer expected it to work.

That's why I find type hinting to be relatively high effort for relatively low benefit and choose to use it only where I feel it makes life better. The optionality suits me. It's a nice to have but that's all.

Re: Python’s “type hints” are a bit of a disappointment to me

#359

Post by Mehdi2277 »

qayxc wrote: post #343
kortex wrote: post #326

This article is so hopelessly mis-informed, that it's practically hard to read without screaming "that's not how any of this works!" Most of their main arguments - type hints can be wrong, they can be ignored, and they don't inform you in a useful way about program state because of this - all evaporate as soon as you start using the correct tooling. My stack is pycharm, mypy, pydantic, sqlalchemy stub, several mypy p…

As long as you don't have to rely on 3rd party libraries, sure, type hints are fine. Many popular libraries, however, don't have type hints or use them inconsistently (looking at you numpy, pytorch, tensorflow, scipy, ...). Always great fun to see function signatures that use type hints for the parameters and return Any . So it really depends what kind of code you write - web frameworks might be fine, data science an…

You can write type stubs for other libraries. Over past year my team had a 100k line codebase that we’ve gotten mostly typed. But we still use a lot of other untyped libraries like tensorflow. So in past couple months we’ve started adding type stubs to the parts of the api we use. While tensorflow/numpy/etc are massive libraries most people only use a small subset of api. You can write type stubs for the subset you use. It definitely takes time but I find it very helpful to have useful type checking of tensorflow so I bit bullet and started writing stubs. I explored upstreaming them to type shed but got lazy on dealing with CI issue although I hope to revisit it later.

Re: Python’s “type hints” are a bit of a disappointment to me

#360

Post by Mehdi2277 »

neves wrote: post #269
jsmeaton wrote: post #239

Adding type hints to Python has increased my productivity by at least a factor of 10. They allow you to reason about code in a local function without having to track back up dozens of call sites to ensure you're getting what you think you're getting. That alone is worth the price of admission. Both when editing code or reviewing someone else's. It's fantastic, particularly in a very large code base. The editor experi…

Nice. How do you force types just in the new code?

My team started with an untyped codebase a year ago. I added a rule to CI that every pr that touches files with type errors must decrease the type errors in touched files by at least 5. When we started we had like 30k type errors. A year later and it’s about 5k left. This was a small wrapper script over mypy/pyright that just called them twice once on master and once on your branch to compare error counts.

I did occasionally get pushback on it but stricter checks will be inconvenience at first and I argued it’d help over time. Now that we’ve had it a year it’s pretty well accepted in same way many would accept run formatter.

Post reply on HN