Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python’s “type hints” are a bit of a disappointment to me

uninformativ.de

Post reply on HN
Print view View original

531–540 of 597 posts

Re: Python’s “type hints” are a bit of a disappointment to me

#531

Post by icedchai »

Too wrote: post #296
neves wrote: post #269

Earlier quoted context omitted.

Nice. How do you force types just in the new code?

Mypy can be configured to ignore modules based on their names. Though honestly I think it’s worth it to just bite the bullet and spend a full day or two to go through and fix every single error. After this you will have a much easier time navigating code base. I mean, your “new code” is probably modifications to your old code or calling your old code right? So you want to have at least the boundary between new and ol…

For any large code base, it'll go on for months, since it gets done between higher priority work. You'll discover some problems, sure. But if they were really important, you'd have hit them earlier. My experience is these sorts of efforts create a lot of busy work and not much more.

Re: Python’s “type hints” are a bit of a disappointment to me

#532

Post by the_af »

rglullis wrote: post #518
the_af wrote: post #516

Earlier quoted context omitted.

> "What I am having trouble to grasp is [...] why do you keep conflating type hinting with type checking?" It's not my business to help you with your trouble grasping things, but why do you think I'm conflating type hinting with type checking? I specifically asked in this comments section what type hinting meant if not what I thought, and was told by several people "it's just standardized comments", which I then expl…

> How about both? Because the "issues" with type hinting that you are talking about are only based on what you wished it was, not on what it is or what it could become! Ask yourself this: what about the python's type hinting story you think could be improved without turning python into a statically typed language? What about the python's type checking story that is missing or broken and that is attributable to the la…

> "Quite frankly, no. My motivation for the conversation now is mostly to see how long is going to take you to realize [...]"

Oh, I see. Well, I have no interest in that kind of conversation.

Goodbye.

Re: Python’s “type hints” are a bit of a disappointment to me

#533

Post by nine_zeros »

Spivak wrote: post #476
nine_zeros wrote: post #470

Earlier quoted context omitted.

My codebase is more complex than str, text, dict, int, bool. So developers are creating complex custom types with union[custom_type_1, custom_type_2] where each custom type could have more unions of other custom types. These custom types then get imported everywhere. It is utter garbage.

I don't follow. So some 3rd party library will define a type like Routes = Union[RouteSet, List[Union[str, Route]]] and this is bad? The complexity is already there, you just don't want to see it?

No, my own devs will write insane nested types at the detriment of all developers who end up looking at that code later.

Routes = Union[GooglyRoutes, MetaRoutes]

Where, GooglyRoutes=Union[List[CloudyRoutes], Sequence[AdseyRoutes]]

Where CloudyRoutes = Union[RouteContainers]

And then they'd import these types all over the python repo creating all kinds of import issues. Fml.

Re: Python’s “type hints” are a bit of a disappointment to me

#534

Post by int_19h »

bayesian_horse wrote: post #290
waitlist wrote: post #285

Earlier quoted context omitted.

I think data science is a perfect example in favor of types -- the code is often terrible because of the lack of typing. Pandas has notoriously poor developer ergonomics, and I recall painfully poring over type errors across the board -- lists, dataframes, numpy arrays, etc. are all iterables, so they can be interchanged in some contexts, but not in others. Had I had MyPy back when I was working in data science, I wo…

What Pandas does is notoriously hard to fit into a compile-time type system. Certainly too hard to go into the brains of scientists who didn't grow up coding. No, the code in data science isn't bad because of the lack of typing. The code is "bad" mostly because those writing it are relatively fresh from starting to program. Also there is more pressure to make things possible, often just to run it once, and neglect re…

> What Pandas does is notoriously hard to fit into a compile-time type system.

Sounds a lot like F# type providers to me.

https://thesharperdev.com/introduction-to-fsharp-type-provid...

Re: Python’s “type hints” are a bit of a disappointment to me

#535

Post by rglullis »

the_af wrote: post #532
rglullis wrote: post #518

Earlier quoted context omitted.

> How about both? Because the "issues" with type hinting that you are talking about are only based on what you wished it was, not on what it is or what it could become! Ask yourself this: what about the python's type hinting story you think could be improved without turning python into a statically typed language? What about the python's type checking story that is missing or broken and that is attributable to the la…

> "Quite frankly, no. My motivation for the conversation now is mostly to see how long is going to take you to realize [...]" Oh, I see. Well, I have no interest in that kind of conversation. Goodbye.

Please don't read it as snark. It was not my intention.

What I am trying to say is that perhaps the lesson you could take from this it's to not think that something is "inferior" or "needs fixing" just because it doesn't immediately meet your expectations.

It isn't Python's fault that you have to use at the job even though you don't like it.

It isn't Python's fault that you have to use even though you are absolutely certain that dynamically typed languages are worse.

It isn't Python's fault that type hints are just hints and can not become a full-fledged type system.

It isn't Python's fault that mypy is still not mature enough to work reliably for you.

And it certainly isn't Python's fault that your coworkers don't care about all that as much as you do.

Re: Python’s “type hints” are a bit of a disappointment to me

#536

Post by int_19h »

qsort wrote: post #378
krageon wrote: post #367

Earlier quoted context omitted.

> Pandas does is notoriously hard to fit into a compile-time type system What, mutate data with known structure? What exactly do you imagine is hard about this, let alone notoriously so?

Not hard per se, but extremely unergonomic. A pandas dataframe types to something like Iterable[SomeKindOfProductType[int, str, str, ... (78 other columns)]]. The formal type of a dataframe in the middle of a transformation is... not very useful to know.

It's not very useful to type, but that's why we have type inference.

It's certainly very useful to know, even if that knowledge is indirect via the type of the resulting dataframe after all the transforms.

Re: Python’s “type hints” are a bit of a disappointment to me

#537

Post by int_19h »

HelloNurse wrote: post #353
arinlen wrote: post #342

Earlier quoted context omitted.

> What Pandas does is notoriously hard to fit into a compile-time type system. Certainly too hard to go into the brains of scientists who didn't grow up coding. I'm not sure if that's true. Doesn't Pandas handle ETL and some anaysis? There is nothing inherent to ETL that makes it a hard problem with compiled languages. In your opinion, what does Pandas do that's hard to do with compile-time languages?

It isn't a matter of "compile time": explicit type declarations and definitions can often be formally sound but practically worthless. Significant types in ETL-style applications typically come from outside (e.g. a certain CSV column in the input file contains a date in YYYYMMDD format, or maybe YYYYDDMM, figure it out, and don't forget time zones or your accounting will go wrong). Then types are mostly complex but o…

The source code shouldn't need to say anything about the type of the resulting matrix explicitly, perhaps. But why shouldn't the type system keep track of shapes and deduce the accurate type for the result of said multiplication?

Re: Python’s “type hints” are a bit of a disappointment to me

#538

Post by int_19h »

wheelerof4te wrote: post #474
animal_spirits wrote: post #471

Earlier quoted context omitted.

If you find that you are returning silly types like that, then it is an indicator you are writing your function badly and should refactor it to be simpler

That's not a silly type, it is just a silly type annotation. It's easy to unpack the tuple inside that list. What is not easy is type-hinting it. What if I want it to be both an list and an iterator? Well, I would have to use Union. I'll leave it to you to write that Union "type".

It is a silly type for a function to return, because you have three levels of data structure nesting, with not one bit of information about the meaning of any of the values except their types. If I call your function and then do result[1][2], what does that mean?

This is exactly the kind of situation where you're supposed to use a named tuple or a data class to make it clear what's what.

Re: Python’s “type hints” are a bit of a disappointment to me

#539

Post by icedchai »

Mehdi2277 wrote: post #360
neves wrote: post #269

Earlier quoted context omitted.

Nice. How do you force types just in the new code?

My team started with an untyped codebase a year ago. I added a rule to CI that every pr that touches files with type errors must decrease the type errors in touched files by at least 5. When we started we had like 30k type errors. A year later and it’s about 5k left. This was a small wrapper script over mypy/pyright that just called them twice once on master and once on your branch to compare error counts. I did occa…

The problem with these sorts of "rules" is they create needless changes outside of the PR's primary focus. The PR's purpose is to do X: add a feature, fix a bug, whatever. Now because I've done X, you want me to do Y, a set of totally unrelated changes that ultimately are a distraction.

Re: Python’s “type hints” are a bit of a disappointment to me

#540

Post by bayesian_horse »

dragonwriter wrote: post #492
bayesian_horse wrote: post #421

Earlier quoted context omitted.

You can anonymize such data or get the necessary agreements for a small subset. All of which is tricky, but not impossible.

> You can anonymize such data ...thereby destroying the production-like features for which you want it for testing, which you then need to recreate and reintroduce, so you might as well just synthesize test data in the first place, since that's what you end up doing anyway, in effect.

I know you probably just want to be annoying, but really, there is a world of difference between completely synthetic and anonymized data. Without talking about specifics, anonymizing is just the operation of making the process of deanonymization a lot of harder. "Hard enough" is usually specified in some form by the regulator. You can identify an individual by their ECG data, for example, it's just really hard...

No, in actual practice you don't scrub the stuff you actually need to test.

Post reply on HN