Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-sourcing MonkeyType – Let your Python code type-hint itself

engineering.instagram.com

Post reply on HN
Print view View original

41–50 of 237 posts

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#41

Post by Cieplak »

Given that people have asked why not use a statically typed language, seems appropriate to mention that it's possible to write pythonic-looking C++:

http://preshing.com/20141202/cpp-has-become-more-pythonic/

I have been using C++ a lot lately but really wish there were more tools for reflection at compile time, e.g., ability to iterate over all the members of a class. Other than that, I'm really loving C++17's auto template parameters and type deduction capabilities, plus code that's 200x faster at runtime than most interpreted languages. I've found autocompletion in CLion to be slightly better than autocompletion PyCharm, but not quite as good as IPython or IntelliJ with Java.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#42

Post by scrollaway »

fiatjaf wrote: post #40
scrollaway wrote: post #27

Earlier quoted context omitted.

Let's make this thread useful instead: Please enlighten the HN community at large: Tell us how you'd build instagram, what your tech choices would be and why Python is the "worst technology choice". (If this goes well maybe you'll get an email from Mike Krieger with the subject line "Here, you do this".)

Do you really think Instagram is a great work of art, that the Instagram codebase is exemplary? I think it could be pure shit -- and indeed a very simple idea. They got big and famous, but it wasn't for dexterity in language choice or coding.

If you believe it to be that simple and low quality a codebase, why does it matter which language they pick?

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#43

Post by walrus1066 »

maltalex wrote: post #13

Reading this as someone who writes mostly in statically typed languages, the whole exercise seems odd. Having so much dynamically typed code to maintain that you need to run production code using a separate tool just to figure out the types sounds just wrong. Why not use a statically typed language for such a large code-base? Is this done by purpose, or did they end up with a million lines of Python code and are look…

Think you trade maintainability for initial productivity with a dynamic language.

You can churn out a greenfield project faster as you don't have to spend upfront time mapping out interfaces, DTO's etc. It's also easier to 'hack'.

Of course the above makes the code harder to maintain and reason with, unless written by very disciplined engineers.

So these languages are a natural fit for startups (and things like prototyping and scripting).

Instagram would have been in this category, and it's worked out well for them.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#44

Post by ambivalence »

JosephRedfern wrote: post #2

This sounds similar to Dropbox's PyAnnotate -- Guido van Rossum writes about it here: http://mypy-lang.blogspot.co.uk/2017/11/dropbox-releases-pya... Would be interesting to see how MonkeyType and PyAnnotate compare.

MonkeyType is Python 3-centric, PyAnnotate is Python 2-centric.

PyAnnotate couples re-applying the types with the tool, and uses type comments for this.

MonkeyType generates .pyi files that you can either use directly or re-apply them to your code as proper type annotations.

Other than that, MonkeyType is used on a daily basis internally and solves a bunch of common annoyances of systems like these, like duplicates in unions, applying better types to stuff that was already hand-annotated with Any, etc.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#45

Post by ambivalence »

sbuccini wrote: post #12
JosephRedfern wrote: post #2

This sounds similar to Dropbox's PyAnnotate -- Guido van Rossum writes about it here: http://mypy-lang.blogspot.co.uk/2017/11/dropbox-releases-pya... Would be interesting to see how MonkeyType and PyAnnotate compare.

I'd also like to hear more about this -- both the feature set and the development process. It's interesting that two large engineering organizations responsible for some of the most popular applications on the planet spent hundreds of engineering hours building almost the same exact tool at around the exact same time. I'd also like to know how much quicker or better it could have been completed if it had been done ou…

The idea to gather types at runtime is as old as PEP 484. The Dropbox and Facebook teams working on Python type checking know each other. We both worked on our implementations independently since we wanted to first test internally whether the idea holds water. For example, I personally thought it wouldn't be as useful in practice as it turned out to be!

We knew we're going to open-source each others' implementations, esp. that Instagram's is focusing solely on Python 3 which isn't useful for Dropbox at the moment. It just took a while to get through the process of open sourcing what we had (cleaning up the early implementation with limited documentation, decoupling from internal data stores, etc.).

Would it be cheaper if this started out in the open? Probably, but I don't think by quite the margin as you expect.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#46

Post by xapata »

fiatjaf wrote: post #37
realbarack wrote: post #24

Earlier quoted context omitted.

I agree with you but I think it's pretty straightforward how this sort of thing happens: 1. Startup builds thing fast in dynamic language because they need to optimize for development speed and iteration, not maintainability or scalability. 2. Startup grows and continues to hire for expertise in the tech stack they are mostly already using. 3. Repeat for some years and some hundreds of engineers and you arrive at thi…

Isn't it faster to write everything in Go, for example, that has a compiler guiding you all the way and you rarely get runtime errors? I feel my developer time very much "optimized" when writing backends in Go than when I wrote then in Python (I also tried Node, which was a disaster).

Empirically? Doesn't seem like it. But it's hard to run a good controlled experiment. The confounding effects of the team members, the project goal, and the vagaries of business are too noisy.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#47

Post by xzel »

azhenley wrote: post #20
maltalex wrote: post #13

Reading this as someone who writes mostly in statically typed languages, the whole exercise seems odd. Having so much dynamically typed code to maintain that you need to run production code using a separate tool just to figure out the types sounds just wrong. Why not use a statically typed language for such a large code-base? Is this done by purpose, or did they end up with a million lines of Python code and are look…

Didn't Facebook do this for PHP (Hack) and Microsoft with JavaScript (TypeScript)? From a technical aspect, I do find these projects cool. I wonder if its more efficient for large companies to initially develop using dynamic languages then transition them with these optionally typed languages.

Facebook did even more with PHP with HHVM and other tooling. I'm not sure if JS was exactly the same for Microsoft as I saw it more as a cool internal project that got big vs something that eventually became necessary.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#48

Post by LrnByTeach »

realbarack wrote: post #24
maltalex wrote: post #13

Reading this as someone who writes mostly in statically typed languages, the whole exercise seems odd. Having so much dynamically typed code to maintain that you need to run production code using a separate tool just to figure out the types sounds just wrong. Why not use a statically typed language for such a large code-base? Is this done by purpose, or did they end up with a million lines of Python code and are look…

I agree with you but I think it's pretty straightforward how this sort of thing happens: 1. Startup builds thing fast in dynamic language because they need to optimize for development speed and iteration, not maintainability or scalability. 2. Startup grows and continues to hire for expertise in the tech stack they are mostly already using. 3. Repeat for some years and some hundreds of engineers and you arrive at thi…

> 1. Startup builds thing fast in dynamic language because they need to optimize for development speed and iteration, not maintainability or scalability.

> 3. Repeat for some years and some hundreds of engineers and you arrive at this exact scenario.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#49

Post by panopticon »

vasilipupkin wrote: post #35
newfoundglory wrote: post #30

Earlier quoted context omitted.

80% of startups are going to die before their codebase reaches this size. They should probably not be making tech stack choices based on "what if we succeed beyond all likelihood".

two things come to mind: maybe those startups would die at a slower rate if their code was comprehensible to begin with ? but leaving that aside, for companies where survival is not an issue then, doesn't this indicate that using a dynamically typed language is not great?

I would be interested in seeing the number of startups that fail due to technical debt. My instinct is that most startups fail for business reasons (no clear need, not enough/right sales, poor management, bad pitch, solving the wrong problem, etc).
Post reply on HN