Why didn't you make the Python 3 type checking advisory instead? Like what Facebook did with Flow and Hack, why not make write a product that lets you statically analyze the types and will never itself cause runtime errors, and transition to types that way? What advantage does building this tool have? I understand that the thing I'm describing involves modifying the way Python 3 handles type annotations, but it doesn't seem like more work than building out all of the instrumentation you've done here.
Open-sourcing MonkeyType – Let your Python code type-hint itself
51–60 of 237 posts
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#52Reading this as someone who writes mostly in statically typed languages, the whole exercise seems odd. Having so much dynamically typed code to maintain that you need to run production code using a separate tool just to figure out the types sounds just wrong. Why not use a statically typed language for such a large code-base? Is this done by purpose, or did they end up with a million lines of Python code and are look…
>Is this done by purpose, or did they end up with a million lines of Python code and are looking for ways to make the maintenance easier? Definitely the latter. I've seen this discussion a few times before, and it's always the same. Your initial developers are not looking down the road to the million lines of code milestone, they're just trying to make a product that might actually make some money here and now. I'm s…
I'm not sure what exactly "Python" has to do with that
Well as he said, a statically typed language is better in that kind of situation because it enables a better class of tooling and the typing system enforces certain style constraints, that enables better quality of code analysis en mass.
Python specifically is very lightweight in this regards with little in the way of naming constraints (vs for instance Ruby having different formatting rules for different types)
So yeah, it’s not “just like any other language” - horses for courses
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#53Similarly, I'm amazed by Pycharm (IDE), which supports pretty good inferred type hints from debugging and static code anlysis, btw. Makes writing code a lot easier. Looking forward to trying MonkeyType, it seems awesome for lager projects.
Pycharm relies on pre-generated annotations as well, they're just built-in - it's not as magically trace-y and infer-y as it might seem at a casual glance.
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#54Fantastic contribution back to the community; I look forward to trying it out.
I must say this is the first time I've been disappointed with the quality of discussion on HN. For a community that promotes using the right tool for the job at the time, I would have thought people would be more open to the choices the early engineers made. I'm sure that Instagram are using a variety of tech across their stack.
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#55Reading this as someone who writes mostly in statically typed languages, the whole exercise seems odd. Having so much dynamically typed code to maintain that you need to run production code using a separate tool just to figure out the types sounds just wrong. Why not use a statically typed language for such a large code-base? Is this done by purpose, or did they end up with a million lines of Python code and are look…
My job is mostly machine learning and statistics. I would love to use something like Haskell or even Java, but the NumPy/SciPy/Scikit-learn/Pandas etc. ecosystem is just so far ahead of everything else that it's not worth it.
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#56Why didn't you make the Python 3 type checking advisory instead? Like what Facebook did with Flow and Hack, why not make write a product that lets you statically analyze the types and will never itself cause runtime errors, and transition to types that way? What advantage does building this tool have? I understand that the thing I'm describing involves modifying the way Python 3 handles type annotations, but it doesn…
I'm not quite sure what you mean. Python has an external static checker for types, it's called mypy.
Python's type annotations are in fact very similar to Flow and Hack in the sense that they provide gradual typing. The specification (see: PEP 484) describes that only annotated functions are type checked. Calls to non-annotated code are treated as accepting any type in arguments and returning the Any type (a special type which effectively silences the type checker).
This generates a chicken and egg problem: if you don't have enough functions annotated, the type checker won't be able to provide meaningful output to you. So convincing people to annotate their code is harder: they don't see the benefit right away. Worse yet, you already have tens of thousands of functions in your code that you know work in production but were written before type annotations were introduced. It's not really feasible to come back and fill this information manually.
MonkeyType is a tool that gathers types at runtime and enables putting them back in your code as annotations. The goal is for mypy to have more information to work with, making it way more useful.
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#57Earlier quoted context omitted.
I agree with you but I think it's pretty straightforward how this sort of thing happens: 1. Startup builds thing fast in dynamic language because they need to optimize for development speed and iteration, not maintainability or scalability. 2. Startup grows and continues to hire for expertise in the tech stack they are mostly already using. 3. Repeat for some years and some hundreds of engineers and you arrive at thi…
Isn't it faster to write everything in Go, for example, that has a compiler guiding you all the way and you rarely get runtime errors? I feel my developer time very much "optimized" when writing backends in Go than when I wrote then in Python (I also tried Node, which was a disaster).
Go wasn't a serious option for Instagram though. According to Wikipedia Instagram launched in 2010, Go launched in 2009. That would have made them very early adopters, was the library support there back then like it was for Python?
Just because there may be better tools now, should they scrap their working code that earned their fortune?
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#58Earlier quoted context omitted.
I agree with you but I think it's pretty straightforward how this sort of thing happens: 1. Startup builds thing fast in dynamic language because they need to optimize for development speed and iteration, not maintainability or scalability. 2. Startup grows and continues to hire for expertise in the tech stack they are mostly already using. 3. Repeat for some years and some hundreds of engineers and you arrive at thi…
Isn't it faster to write everything in Go, for example, that has a compiler guiding you all the way and you rarely get runtime errors? I feel my developer time very much "optimized" when writing backends in Go than when I wrote then in Python (I also tried Node, which was a disaster).
We're getting into a world where languages finally have type systems that dont suck for fast development. This wasn't the case until very recently. And many of the current options only became realistically viable in the last few years (or months!).
We still have to work with the world as it exists. Not as it should be or will be. And even with the crop of modern languages, its often still faster to start with less optimal languages and fix shit in the 1/10 chance you're actually successful.
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#59Reading all of the comments from engineers who seem to either posess a time machine to send current tech back in time, or are criticising the technical choices that made the founders $squillions, is making me a bit mad. As a diversion perhaps some of them could list a few billion dollar startups that made perfect choices at the start and never had any cause to refactor or reimplement code as they grew?
Re: Open-sourcing MonkeyType – Let your Python code type-hint itself
#60Reading this as someone who writes mostly in statically typed languages, the whole exercise seems odd. Having so much dynamically typed code to maintain that you need to run production code using a separate tool just to figure out the types sounds just wrong. Why not use a statically typed language for such a large code-base? Is this done by purpose, or did they end up with a million lines of Python code and are look…
I really don't get people. Instagram and Dropbox, through typing annotations in Python, are gradually improving a language that has codebases running globally, from YouTube to NASA. Clearly something is right with the situation when the incentives are aligned for a tech company to contribute back to the open source community in such fundamental ways. So why look for the mole and think "They should have done it differ…
No, it's like asking someone who spent a lot of time building an octagonal wheel and is now trying to shave down the corners... why didn't you use a circle to begin with.