Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Instagram Makes a Smooth Move to Python 3

thenewstack.io

Post reply on HN
Print view View original

51–60 of 62 posts

Re: Instagram Makes a Smooth Move to Python 3

#51

Post by crdoconnor »

mahyarm wrote: post #11
dmalvarado wrote: post #6

“Yeah, Python is great in so many ways, too bad it’s not really scalable.” I'm not even sure what this means anymore. I guess I'm just not sure how any language, when used correctly, could be inherently unscalable. My guess is statements like this came from a time when monoliths were the application design of choice? Now, assuming Instagram has just 1,000 photo handling servers, each one is only responsible for 95,00…

The threading story isn't great. It's not statically typed, which can make working on a large codebase with more people more painful. It also forces you to write more unit testing code coverage to make up for the lack of a static compiler checking things for you. Raw performance is not good compared to golang or java. When you start getting to a certain scale, developers are cheaper than your server costs in some cas…

"It's not statically typed, which can make working on a large codebase with more people more painful. It also forces you to write more unit testing code coverage to make up for the lack of a static compiler checking things for you."

I never missed static typing on large code bases but I had numerous bugs caused by python's implicit type conversions - string to iterable of characters ["h", "e", "l", "l", "o"], None to False, string "no" to True, 0 to False, "" to False, etc.

A lot of this mirrors C's infuriating implicit type conversions.

Re: Instagram Makes a Smooth Move to Python 3

#52

Post by crdoconnor »

mahyarm wrote: post #17
bpicolo wrote: post #14

Earlier quoted context omitted.

And at that scale you can make additional microservices for the critical path where it matters.

Not really. You'll notice a pattern in most tech bigcos where they move from dynamic lang X to a statically typed language that can multithread properly. A few examples: Ruby on rails to java (twitter). Java & c++ (google). Java (linked in). Python/Node -> Java/Go (uber). Or they start doing silly things like make a new VM (facebook).

From python to go and back again https://news.ycombinator.com/item?id=10402307 (mozilla)

Re: Instagram Makes a Smooth Move to Python 3

#53

Post by nailer »

stock_toaster wrote: post #36
stuartaxelowen wrote: post #22

Earlier quoted context omitted.

I heard through the grapevine that Go was only chosen at Uber to help with hiring.

That seems silly. There are far more Java devs out there than Go devs. Also, I would assume Uber probably has hiring problems in general at this point.

There are more good Go devs. A lot of Java programmers are enterprise.

Re: Instagram Makes a Smooth Move to Python 3

#54

Post by MatthewWilkes »

I watched the PyCon keynote on this topic, and while it's nice to hear they've moved to Python3 their approach probably shouldn't be copied.

For example, in their codebase they had ambiguity between bytestrings and unicode strings. As Python3 tries to prevent you doing this, to resolve a big footgun from Python2.

The right fix here is to be consistent in your use of strings. Sometimes that is tricky because of how third party libraries have decided to implement their 2/3 compatibility, but it helps prevent shooting yourself in the foot with unicode bugs down the line.

Instagram did not do this. They created utility functions to force their data into the format they wanted at the point it is used. In other places they used tuple() to make sure that map calls that had side effects were fully iterated over.

In short, they had bad Python2 code and now have had Python3 code. Sometimes, at large scale, it's your only choice. But to smaller companies looking at this it's a bad idea. You're setting a precedent in your code that it's okay to make the same mistakes that Python3 tried to prevent.

Re: Instagram Makes a Smooth Move to Python 3

#55

Post by MatthewWilkes »

vhost- wrote: post #32
mixmastamyk wrote: post #9

Right, you hear a lot of griping about moving from Python 2 to 3 but I personally didn't have as much trouble as expected. Some of my projects just worked. One small tip I don't think they mentioned. Start using the logging module instead of print and it will eliminate one class of potential issues.

I've heard people complain about print being a function now more than a hand full of times and my response is always "are you really using print that much in your code base?"

You still use it in debuggers and in the REPL. Using it seldom makes it harder to relearn the muscle memory, and only hitting it when you're debugging makes it more likely that you're already frustrated when it happens.

Re: Instagram Makes a Smooth Move to Python 3

#56

Post by shalabhc »

bpicolo wrote: post #49
shalabhc wrote: post #44

Earlier quoted context omitted.

There is also Google's pytype: https://github.com/google/pytype

That looks like it adds types rather than checks?

It does both. I can do some inference but definitely does type checking. Here's a Pycon talk from last year about pytype: https://youtu.be/IDm_YIQihhs

Re: Instagram Makes a Smooth Move to Python 3

#57

Post by bpicolo »

shalabhc wrote: post #56
bpicolo wrote: post #49

Earlier quoted context omitted.

That looks like it adds types rather than checks?

It does both. I can do some inference but definitely does type checking. Here's a Pycon talk from last year about pytype: https://youtu.be/IDm_YIQihhs

It needs severely more documentation, heh. Why would I use it over mypy, which appears to have a lot more community investment, thorough docs, and is somewhat official? It also doesn't support python3.6 so it doesn't know about format strings =/

Re: Instagram Makes a Smooth Move to Python 3

#58

Post by treebog »

bpicolo wrote: post #41
mahyarm wrote: post #17

Earlier quoted context omitted.

Not really. You'll notice a pattern in most tech bigcos where they move from dynamic lang X to a statically typed language that can multithread properly. A few examples: Ruby on rails to java (twitter). Java & c++ (google). Java (linked in). Python/Node -> Java/Go (uber). Or they start doing silly things like make a new VM (facebook).

Google uses plenty of Python, no? I don't think threading is a relevant factor for many of those decisions. Perf and static typing, sure.

Not a googler, but I have heard that most of the python at google is under Youtube, which Google got in an acquisition.

Re: Instagram Makes a Smooth Move to Python 3

#59

Post by brianwawok »

crdoconnor wrote: post #51
mahyarm wrote: post #11

Earlier quoted context omitted.

The threading story isn't great. It's not statically typed, which can make working on a large codebase with more people more painful. It also forces you to write more unit testing code coverage to make up for the lack of a static compiler checking things for you. Raw performance is not good compared to golang or java. When you start getting to a certain scale, developers are cheaper than your server costs in some cas…

"It's not statically typed, which can make working on a large codebase with more people more painful. It also forces you to write more unit testing code coverage to make up for the lack of a static compiler checking things for you." I never missed static typing on large code bases but I had numerous bugs caused by python's implicit type conversions - string to iterable of characters ["h", "e", "l", "l", "o"], None to…

For what it is worth string to iterable char bugs happen to me plenty in scala. Hard problem to fix. Array of strings or string of chars?

Re: Instagram Makes a Smooth Move to Python 3

#60

Post by crdoconnor »

brianwawok wrote: post #59
crdoconnor wrote: post #51

Earlier quoted context omitted.

"It's not statically typed, which can make working on a large codebase with more people more painful. It also forces you to write more unit testing code coverage to make up for the lack of a static compiler checking things for you." I never missed static typing on large code bases but I had numerous bugs caused by python's implicit type conversions - string to iterable of characters ["h", "e", "l", "l", "o"], None to…

For what it is worth string to iterable char bugs happen to me plenty in scala. Hard problem to fix. Array of strings or string of chars?

I don't think it is that hard. Simply stop making strings implicitly iterable and make chars a different type.
Post reply on HN