Live data from Hacker News

How we rolled out one of the largest Python 3 migrations

blogs.dropbox.com

51–60 of 141 posts

Re: How we rolled out one of the largest Python 3 migrations

#51
post #18

Interesting write-up, though it leaves me terrified how a relatively small, more or less single purpose application like the Dropbox client has over 1 million lines of code in Python alone.

The size of a codebase is less about its function and more about the number of people adding code to it.

In turn this becomes the number of people required to maintain a codebase.

Re: How we rolled out one of the largest Python 3 migrations

#52
Great write-up and those two graphs are interesting. It's cool to learn how different companies treat their beta users. I wish this article touched upon more of the technically difficulties with switching from Python 2 to 3 too.

A̵l̵s̵o̵ ̵a̵ ̵s̵m̵a̵l̵l̵ ̵h̵e̵a̵d̵s̵-̵u̵p̵:̵ ̵a̵t̵ ̵t̵h̵e̵ ̵e̵n̵d̵ ̵o̵f̵ ̵t̵h̵e̵ ̵f̵i̵r̵s̵t̵ ̵p̵a̵r̵a̵g̵r̵a̵p̵h̵,̵ ̵"̵v̵e̵n̵e̵r̵a̵b̵l̵e̵"̵ ̵s̵h̵o̵u̵l̵d̵ ̵b̵e̵ ̵"̵v̵u̵l̵n̵e̵r̵a̵b̵l̵e̵"̵ ̵(̵u̵n̵l̵e̵s̵s̵ ̵y̵o̵u̵ ̵m̵e̵a̵n̵ ̵w̵e̵ ̵s̵h̵o̵u̵l̵d̵ ̵l̵o̵o̵k̵ ̵b̵a̵c̵k̵ ̵a̵t̵ ̵t̵h̵e̵ ̵s̵a̵c̵r̵e̵d̵ ̵p̵y̵w̵i̵n̵3̵2̵ ̵l̵i̵b̵r̵a̵r̵y̵ ̵w̵i̵t̵h̵ ̵h̵o̵n̵o̵r̵)̵.̵

Re: How we rolled out one of the largest Python 3 migrations

#53

Great write-up and those two graphs are interesting. It's cool to learn how different companies treat their beta users. I wish this article touched upon more of the technically difficulties with switching from Python 2 to 3 too. A̵l̵s̵o̵ ̵a̵ ̵s̵m̵a̵l̵l̵ ̵h̵e̵a̵d̵s̵-̵u̵p̵:̵ ̵a̵t̵ ̵t̵h̵e̵ ̵e̵n̵d̵ ̵o̵f̵ ̵t̵h̵e̵ ̵f̵i̵r̵s̵t̵ ̵p̵a̵r̵a̵g̵r̵a̵p̵h̵,̵ ̵"̵v̵e̵n̵e̵r̵a̵b̵l̵e̵"̵ ̵s̵h̵o̵u̵l̵d̵ ̵b̵e̵ ̵"̵v̵u̵l̵n̵e̵r̵a̵b̵l̵e̵"̵ ̵(̵u̵n…

Venerable is correct here.

Re: How we rolled out one of the largest Python 3 migrations

#54
post #42

Earlier quoted context omitted.

What would you consider the main cause(s) of that?

The job of a programmer is to write code, so that’s what they do. I doubt there are very many people at Dropbox whose job it is to remove code!

I dunno - refactoring often involves removal of a fair bit of code

Re: How we rolled out one of the largest Python 3 migrations

#55
post #48

> However, rather than use the native toolchains directly, such as Xcode for macOS, we delegated the creation of platform-compliant binaries to py2exe for Windows, py2app for macOS, and bbfreeze for Linux. I wish the authors of more Python tools would deploy standalone applications. I do not like having to maintain various sets of Python installers/package managers (because every Python tool seems to use a different…

This is possible and actually pretty trivial.

You install all 3rd party dependencies into some directory. The command line entry point is then a simple BASH script which sets the PYTHONPATH to the appropriate installation location and then does the appropriate exec call.

You then have a functionally portable python installation.

Re: How we rolled out one of the largest Python 3 migrations

#56
post #36

Earlier quoted context omitted.

I'm surprised 10% wasn't enough - 10% bottom-line improvement in programming language implementation is normally massive. Twitter is singing from the roof-tops about 10% improvement in Java performance from the new Graal JIT compiler.

PyPy is already way better than that, and has a Python 3.5 implementation.

I'm by no means an expert on the subject, but isn't there an advantage to having a JIT compiler use LLVM?

On another note, I believe one thing that has been problematic for pypy adoption is that it does not automatically work with C extensions or Cython, and generally if someone already had performance issues with CPython, they would have written some C/Cython extensions?

Re: How we rolled out one of the largest Python 3 migrations

#57
post #53

Great write-up and those two graphs are interesting. It's cool to learn how different companies treat their beta users. I wish this article touched upon more of the technically difficulties with switching from Python 2 to 3 too. A̵l̵s̵o̵ ̵a̵ ̵s̵m̵a̵l̵l̵ ̵h̵e̵a̵d̵s̵-̵u̵p̵:̵ ̵a̵t̵ ̵t̵h̵e̵ ̵e̵n̵d̵ ̵o̵f̵ ̵t̵h̵e̵ ̵f̵i̵r̵s̵t̵ ̵p̵a̵r̵a̵g̵r̵a̵p̵h̵,̵ ̵"̵v̵e̵n̵e̵r̵a̵b̵l̵e̵"̵ ̵s̵h̵o̵u̵l̵d̵ ̵b̵e̵ ̵"̵v̵u̵l̵n̵e̵r̵a̵b̵l̵e̵"̵ ̵(̵u̵n…

Venerable is correct here.

You're right, I haven't seen it used outside of a religious context before.

Re: How we rolled out one of the largest Python 3 migrations

#58
post #55
post #48

> However, rather than use the native toolchains directly, such as Xcode for macOS, we delegated the creation of platform-compliant binaries to py2exe for Windows, py2app for macOS, and bbfreeze for Linux. I wish the authors of more Python tools would deploy standalone applications. I do not like having to maintain various sets of Python installers/package managers (because every Python tool seems to use a different…

This is possible and actually pretty trivial. You install all 3rd party dependencies into some directory. The command line entry point is then a simple BASH script which sets the PYTHONPATH to the appropriate installation location and then does the appropriate exec call. You then have a functionally portable python installation.

Isn't this just a hacky virtualenv? 'python3 -mvenv dir && ./dir/bin/pip install requirements'?

Re: How we rolled out one of the largest Python 3 migrations

#59

Interesting write-up, though it leaves me terrified how a relatively small, more or less single purpose application like the Dropbox client has over 1 million lines of code in Python alone.

Especially when you could build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem!

Sarcasm doesn't translate well on the internet, so I'm really not sure if suggesting using CVS (of all things) over a mounted FTP share as a replacement for Dropbox is a joke!

Re: How we rolled out one of the largest Python 3 migrations

#60
post #36

Earlier quoted context omitted.

PyPy is already way better than that, and has a Python 3.5 implementation.

I'm by no means an expert on the subject, but isn't there an advantage to having a JIT compiler use LLVM? On another note, I believe one thing that has been problematic for pypy adoption is that it does not automatically work with C extensions or Cython, and generally if someone already had performance issues with CPython, they would have written some C/Cython extensions?

PyPy is far more than just 'python with a JIT'.

It's a tool that allows you to write an entire interpreter in RPython (a subset of Python) and then have it build a native binary with a free jit compiler included, with the specifics of your language encoded within. The reference implementation for this project is a Python interpreter.

It's seriously, seriously cool.

Post reply on HN