Python types have an expectations problem
1–10 of 115 posts
Re: Python types have an expectations problem
#2There are some other warts also, like hard-to-grok errors if you're using typing syntax that's not yet supported in your version of python.
Like:
somevar: dict[int, int] = {0: 0, 1: 1}
Re: Python types have an expectations problem
#3Summarizing quote:
“Python type hints are a core part of the language, they even have standard library modules (typing), and yet they don’t do anything when used in that language without some external tooling.
That, to me, is a bit of an expectations mismatch.”
I.e. they’re complaining that a type checker like mypy isn’t run by default.
Re: Python types have an expectations problem
#4There are some other warts also, like hard-to-grok errors if you're using typing syntax that's not yet supported in your version of python. Like: somevar: dict[int, int] = {0: 0, 1: 1} Produces "TypeError: 'type' object is not subscriptable"
Yep, it's a good catch.
Hard-to-grok errors tend to appear for every language and every backwards-compatible change unless the language itself specifies the target version in the source code file itself. The only language I know is doing this is Solidity.
Re: Python types have an expectations problem
#5Summarizing quote: “ Python type hints are a core part of the language, they even have standard library modules (typing), and yet they don’t do anything when used in that language without some external tooling. That, to me, is a bit of an expectations mismatch. ” I.e. they’re complaining that a type checker like mypy isn’t run by default.
It's also good to note that all popular editors and IDEs like Visual Studio Code and PyCharm support type hints quite well by default. Some external tooling needed, but it's your editor and you are going to have any case.
Re: Python types have an expectations problem
#6A note; typescript does nothing to ensure types are correct at runtime. Especially in the browser. You need to do runtime checking even if you're using typescript.
In Python, yeah, they're called Type _Hints_ for a reason. Don't count on them at runtime here either.
Both are dynamic languages, it's hard doing anything meta/schema driven with rigid types. If you really want a hard type system, just move to GO or Rust or C or something with a real type system enforced.
Re: Python types have an expectations problem
#7So basically all they want is a wrapper for python which checks first with mypy if a script should be executed, and throws an error otherwise?
Re: Python types have an expectations problem
#8From my experience working on an older Python codebase, this issue is definitely a headache. It's extremely difficult to gradually adopt typing in an older Python codebase with almost no typing information because the only real "enforcement" option seems to be a CI pipeline running something like `mypy`.
This issue compounds in a painful way. Because 99% of your codebase is starting out untyped, you have a couple of options, neither of which I have found to be practically very useful.
For the first option, you can run a blanket `mypy` invocation on your entire codebase and have a massive blast of errors you ignore for some time. Because it's necessarily going to error in the beginning, you can't really fail your CI pipeline as a result of this yet. If you can convince your team to gradually improve typing or set a deadline for eventual CI failure based on types, you might be able to move the needle and eventually get your codebase typed.
From my experience though, this basically just became a CI step everyone ignored, and for people on the team not passionate about typing, they never worried about it.
The other option, which is far more annoying in practice, is to pick a few "seed" files in your codebase that you can add typing information to quickly. Then, supplement your `mypy` invocation with a list of these seed files so it becomes something like `mypy file1.py file2.py ...`
As you continue to improve typing "at the edges" of your codebase, you gradually add more and more files to the `mypy` invocation until you're eventually (hopefully) adding entire subfolders, and then maybe eventually the entire codebase. Starting at the edges means you can enforce the CI check from the beginning and get value quickly.
The issue here is mostly remembering to continue to add files to the `mypy` invocation, which means you're constantly altering your CI pipeline. You know how when you alter a CI command it breaks sometimes because you got the encantation slightly wrong? Multiply this effect across basically every member of your team 1x a week because most people probably haven't edited your CI pipeline before. With even a small team (~7-10) making constant changes to a codebase, this quickly becomes extremely painful, and pipeline failures start eating a significant chunk of time just trying to debug if the encantation is wrong or if the types are actually broken.
We mitigated this by having only one dev add new files to the `mypy` invocation, which worked well for the CI side of the story.
The local side of the story is what ultimately led to enough fatigue to give up. It was hard to get in the rhythm of using local `mypy ...` invocations to check your types as you made changes, and so the experience for most of our team was to push changes, and then the types would break in CI, which was frustrating. They'd go in and try to fix it, and sometimes Python typing gets weird, and a fix wasn't immediately obvious. Eventually you get to `#type: ignore` or `Any`s being thrown around to sidestep the CI pipeline, and your typing story has collapsed again. The real kicker for us was the painful juxtaposition between `mypy` and `import`s. Is the giant swath of errors I'm seeing from this file or from a file I imported? Asking the entire team to become Python typing gurus to sort out these issues was a non-starter.
Does anyone have experience gradually adopting Python typing in a large, older codebase successfully? If so, would you mind sharing the methodology you found success with?
Re: Python types have an expectations problem
#9"I’d have to set up some CI with the type checking step and make it impossible to deploy the code that doesn’t pass it, maybe."
Yes. That's how people successfully use type checking in Python.
Re: Python types have an expectations problem
#10There are some other warts also, like hard-to-grok errors if you're using typing syntax that's not yet supported in your version of python. Like: somevar: dict[int, int] = {0: 0, 1: 1} Produces "TypeError: 'type' object is not subscriptable"
For context, if others aren't familiar: type hinting generics were added to Python 3.9 by PEP 585 (also available in Python 3.7+ with the annotations future import). PEP 484 previously added type hints to Python 3.5 as explicit imports from the stdlib's typing module.