Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-sourcing MonkeyType – Let your Python code type-hint itself

engineering.instagram.com

Post reply on HN
Print view View original

81–90 of 237 posts

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#81

Post by [deleted] »

miohtama wrote: post #75
imiric wrote: post #63

Earlier quoted context omitted.

To add to this, note that type hinting is quite a new feature in Python (introduced in v3.5, released in 2015), and this functionality simply wasn't available before. So any company heavily invested in Python today obviously wants to improve their runtime reliability, without having to rewrite parts of their stack. Stricter typing goes a long way to achieve this, and gradual typing allows you to upgrade the code base…

Dropbox is very heavily invested in Python. I am under impression they hired Guido van Rossum to do exactly this, among other things. First 100% statically type the old codebase, then port it to Python 3. You can statically type Python 2 codebases, but the language does not offer native support for it. Thus, all needs to go to docstrings or comments.

[deleted]

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#82

Post by david-cako »

hacker_9 wrote: post #4

"At Instagram we have hundreds of engineers working on well over a million lines of Python 3." It always amazes me that some of the most popular products around are built with the worst technology choices. And now they had to build their own static type checker, which slows down random samples of real users, just to shore up the language's weaknesses? Outstanding.

Another comment in this thread makes a very compelling point: perhaps there is some wisdom to using a language that can be initially dynamically typed, and moved over to static typing later on.

I don't know. I suspect being half-pregnant is a far bigger issue than the technical debt of dynamic typing, whether it stems from being slower to implement new features, spending time choosing the "correct" tech stacks, or spending more time/money hiring qualified engineers.

There's something to be said for the fact that many of the most successful startups in the world are saturated partly or fully with Python and PHP: getting features and new hires off the ground in as little time and money possible is worth its weight in gold.

I've always been very much of the opinion that the smartest way to go is making good use of FFI where it counts, and doing everything else in whatever gets code out the door most effectively. This TypeScript/Hack/MonkeyType trend just seems like a very welcome extension of that.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#83

Post by david-cako »

chc wrote: post #77
grandmczeb wrote: post #7

Earlier quoted context omitted.

If lots of successful projects use what you consider the “worst” technology, perhaps the problem is with your perception rather than the technology?

That seems like a pretty dodgy metric. Lots of successful construction projects used asbestos, but it is still widely considered a mistake.

The downside to using a certain tech stack is technical debt, not a health risk. Technical debt seems like a wise acquisition in moderation, especially for a startup that wants pictures and buttons on screens ASAP, and even more especially when everyone is realizing that it can rather easily be undone when/where it matters with type checker addons and FFI.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#84

Post by jw- »

imiric wrote: post #63
nawgszy wrote: post #33

Earlier quoted context omitted.

>Is this done by purpose, or did they end up with a million lines of Python code and are looking for ways to make the maintenance easier? Definitely the latter. I've seen this discussion a few times before, and it's always the same. Your initial developers are not looking down the road to the million lines of code milestone, they're just trying to make a product that might actually make some money here and now. I'm s…

To add to this, note that type hinting is quite a new feature in Python (introduced in v3.5, released in 2015), and this functionality simply wasn't available before. So any company heavily invested in Python today obviously wants to improve their runtime reliability, without having to rewrite parts of their stack. Stricter typing goes a long way to achieve this, and gradual typing allows you to upgrade the code base…

I'd consider those languages optionally typed, rather than gradually typed, as they don't insert run-time type checks. Regardless, I still think it's all interesting and valuable work.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#85

Post by muglug »

troels wrote: post #5

Interestingly, I made a similar tool for php some time ago. It was recently revived and is now in active development to bring it up to speed with recent developments in the language. https://github.com/troelskn/phpweaver

That is fascinating. I’m the author of a static analysis tool for PHP that can generate types, but clearly not at the same level as runtime analysis. I’ll use it on my company’s codebase and report back.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#86

Post by [deleted] »

fiatjaf wrote: post #37
realbarack wrote: post #24

Earlier quoted context omitted.

I agree with you but I think it's pretty straightforward how this sort of thing happens: 1. Startup builds thing fast in dynamic language because they need to optimize for development speed and iteration, not maintainability or scalability. 2. Startup grows and continues to hire for expertise in the tech stack they are mostly already using. 3. Repeat for some years and some hundreds of engineers and you arrive at thi…

Isn't it faster to write everything in Go, for example, that has a compiler guiding you all the way and you rarely get runtime errors? I feel my developer time very much "optimized" when writing backends in Go than when I wrote then in Python (I also tried Node, which was a disaster).

[deleted]

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#87

Post by jw- »

CGamesPlay wrote: post #70
ambivalence wrote: post #56

Earlier quoted context omitted.

I'm not quite sure what you mean. Python has an external static checker for types, it's called mypy. Python's type annotations are in fact very similar to Flow and Hack in the sense that they provide gradual typing. The specification (see: PEP 484) describes that only annotated functions are type checked. Calls to non-annotated code are treated as accepting any type in arguments and returning the Any type (a special…

Sorry, I guess I jumped to the conclusion that you profiled in production because you had a mandatory type checker which would cause runtime errors if invalid types were passed. Let me try another question. Do you think that a project like MonkeyType would substantially aid adding flow annotations to a JavaScript project? If so, why is the value to the Python ecosystem higher than the value to the Flow/JavaScript eco…

It's already been flirted with in JS I believe [1], though not specifically for Flow. The problem is that it falls down in the higher order case, which happens quite alot in JS. Also, I don't think JS has a mechanism like sys.setprofile that deals with alot of the pain points.

[1] https://medium.com/fhinkel/runtime-type-information-for-java...

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#88

Post by depressedpanda »

angelsl wrote: post #73
ageofwant wrote: post #65

Earlier quoted context omitted.

They did not spend a lot of time building an octagonal wheel, they built a billion dollar company using Python. Now, when the code base has been proven, and the business rules solidified, they retrofit what they believe will make the code base easier to maintain. Python is an excellent enabler of this kind of dynamic system evolution.

You've totally missed the analogy. See pfranz's comment.

No, I the analogy just wasn't very apt.

Python allowed them to build a successful company. Now, when their stack is mature and maintenance is more important than rapid prototyping, Python allows them to add type hinting.

Because they are engineers, they built a tool (in Python) that allows them to do it in an automated manner.

And all of this is great!

They are evolving their code to fit their needs; it's nothing like making a octagonal wheel and wishing you'd have gone for a round one in the beginning.

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#89

Post by [deleted] »

mbid wrote: post #78
jimnotgym wrote: post #59

Reading all of the comments from engineers who seem to either posess a time machine to send current tech back in time, or are criticising the technical choices that made the founders $squillions, is making me a bit mad. As a diversion perhaps some of them could list a few billion dollar startups that made perfect choices at the start and never had any cause to refactor or reimplement code as they grew?

It's not obvious (and IMO somewhat doubtful) that it was the technical choices of the founders that made them successful. Their choices could well have been bad, just not bad enough to make their business fail.

[deleted]

Re: Open-sourcing MonkeyType – Let your Python code type-hint itself

#90

Post by chc »

david-cako wrote: post #83
chc wrote: post #77

Earlier quoted context omitted.

That seems like a pretty dodgy metric. Lots of successful construction projects used asbestos, but it is still widely considered a mistake.

The downside to using a certain tech stack is technical debt, not a health risk. Technical debt seems like a wise acquisition in moderation, especially for a startup that wants pictures and buttons on screens ASAP, and even more especially when everyone is realizing that it can rather easily be undone when/where it matters with type checker addons and FFI.

That doesn't have anything to do with the question of whether "raw number of successful projects using it" is a good metric for how good something is. Lots of successful companies have made mistakes, and there are even mistakes that many companies in a field made simultaneously (e.g. the XML mania of the early 2000s is not fondly looked back on).

I'm not commenting on the specific issue of whether Python creates good trade-offs, just that I think tallying successful companies is a poor measure of a programming language.

Post reply on HN